We have some nice progress...

Have been able to get the code to a point where I can navigate the folders in a share and also open and display a file. Yes, progress.

And then I tried to create a file... Oops! Well, not really surprising that code path has not been excised yet and many issued need to be checked. Looks like an issue with some of the talloc code possibly - that was the comment in the log file. So will dive into that soon.

Nice to see the progress and get the reward of seeing the navigation and the display.

Bill.

Well, the fact that they do not fall over and crash immediately is more like the truth. I have successfully compiled and linked an initial incantation of Samba 4.3.6 on OpenVMS I64 8.4.

Now the fun stuff has started! Looking through the reams of log files - thank you Samba developers for DEBUG - and finding errors. And then fixing those errors and going to the next iteration of the port. Also as I work through this then I also will continue to turn on more and more options along the way.

It is still very early in this port. There are thousands of files; some will never need to be looked at initially, thank goodness, as they will just compile and run. Others have already been taking time and finding bugs in the OpenVMS CRTL - nothing new there. Thankfully there have been some workarounds already available - except for a tweak here and an tweak there.

One major point that needs to be considered - until there is a fix for CRTL bug dealing with AF_UNIX sockets and bind() - is the "collision" of routines in a couple routines that deal with IO on VMS to support unix/linux style IO - the issue of open(), close(), select(), poll(), bind(), socket() and such that need to play with all devices - terminals, sockets (of all types), pipes, etc. I am hoping to avoid this with having isolated calls, but it may need to be unified - and then of course torn apart to some extent when/if the CRTL fix is released. But of course the issue of supporting older versions of VMS plays there as well.

Anyway, off to a side project while Samba builds in the latest fixes I have implemented.

Have started testing and debugging with the code that runs on OpenVMS.

Have been working through issues with logic and concept - not necessarily final concept/configuration but enough to let code run.

Am not further along than I was awhile back before starting the implementation of the VMS specific framework into the Samba environment. I have read the smb.conf file and am starting to execute code past that setting up the Samba environment.

I have fixed some issues in the vms_security module which did not make sense as far as the initial operation of Samba and updated some issues I saw as well before starting the nightly build.

Yes, nightly build. The build currently takes about 8 hours on VMS on a rx2600 running OpenVMS 8.4. While I have not gone and started any analysis to determine WHERE this cost in time is occurring I have my suspicions.

The cost of creating a process in OpenVMS is fairly expensive. More than opening a file by many times. The cost of image activation is expensive as well. And then I am using Python and Perl; they are part of the build environment, and to not use them is more expensive in terms of re-engineering the build process. But to find a way to get more performance out of them would be very useful.

And of course the issues of how GNV works on OpenVMS - that too is a potential issue as far as performance. For example, to get Python or Perl to properly invoke GNV/Bash as an exec'd process requires the invocation of bash reading a script file which then has the necessary commands. So given a build from GNV the following tree is to be expected:

DCL->bash->python->bash->(cc/ld)->DECC

That is sort of the minimal view

DCL->bash->python->Dbash->python->bash->(cc/ld)->DECC

or even:

DCL->bash->python->bash->python->bash->perl->bash->(cc/ld)->DECC

So we can have from 5 to 10 or more processes invoked at any one time as part of the process of executing the build process associated with the WAF build environment implementation on OpenVMS. The issue being the need to create a DCL based process to them invoke a new image of bash and have it then execute a script file. If there were a way to create a process where the "command interpreter" was bash rather than DCL then that could effectively eliminate as many as three process creations in a chain of processes. It could also eliminate the creation of two or three temporary files used to create the required temporary scripts to drive the process.

Another saving could come about if we could create a mechanism of reusing processes in this environment. That is to say that at least we might be able to eliminate one of the process creations by having the initial invocation of bash from Python as being persistent. It is not clear how one might make this happen or even how one might extend this to other levels of the process tree at present.

While these issues and ruminations are interesting the more interesting is that we are moving forward with the port...

Check back occasionally...

Thanks!

Best,
Bill.

Well, started getting some VMS specific code introduced. Taking advantage (well starting with) prior work which has handled some of the necessary functions previously. Have gotten to a point where I hope in a day or so to be getting back to testing rather than just the rough cuts necessary to get rid of mis-defined items in the build and undefined symbols.

This first set of functions will give a starting point.

Then I expect it will be go one step at a time. See what fails. See what exists in prior work. Adapt from old and or craft anew.

Bill.

https://drive.google.com/open?id=0B8fhl85GHPz2cnlaOXdDdGUtUG8 (VGIT2.ZIP)

Note that the tool is for OpenVMS ia64 only at this time, although it should of course be possible to build it for Alpha 8.4. Also, note that the ZIP file was created on OpenVMS, so I would suggest extracting the contents on a VMS system!

The contents of the ZIP file are as follows:

Archive: SYS$SYSDEVICE:[CAMERON]VGIT2.ZIP;1
Length Date Time Name
--------- ---------- ----- ----
6881280 01-17-2016 20:09 vgit2.exe
1367 01-13-2015 15:39 cert.pem
--------- -------
6882647 2 files

Vgit2.exe is the UNIX-style version of the tool. As I think I mentioned previously, there is a little more functionality available in the VMS CLI-based version, but any such additional functionality is quite minimal and (probably) likely to be little-used. The UNIX-style version is also a little more “user friendly” in that it contains some build-in usage notes (more on this later).

The certificate file (cert.pem) is required if you are going to be interacting with some service like GitHub or BitBucket.

Anyway, to get started, extract the contents of the ZIP file and copy the files into your preferred location. Next, perform the following tasks (or add the commands to login.com, or whatever):

• Define a foreign command for vgit2:

$ vgit2 :== $device:[path]vgit2.exe

• Define the logical name git$ssl_certs to point to the certificate file:

$ define git$ssl_certs device:[path]cert.pem

Next you’ll need to run the following command to set up some basic configuration details:

$ vgit2 config user -n username -e youremail


After this command completes, you should see in your login directory the file git.config, and it should contain the details that you specified in the above command. It is better to create this file before doing anything else to avoid being told by the program to do this the first time you come to do a push. Note that the values you specify are not particularly critical as they can be overridden if/when necessary; however if you will primarily be interacting with some service such as GitHub then you should specify your username for that service and the associated email address.

As mentioned above, the UNIX-style version has some level of built-in usage notes (which saves me typing a whole pile of notes on how to use the thing). For example, running the program without specifying a command or specifying an invalid command, you will get the following output:

$ vgit2
Usage: EXEC14$DKA200:[CAMERON.vgit]vgit2.exe;9 <command> [options] [arguments]
Command summary:
add [-s] [-v] [-u] <file> ...
checkout [-s safe | create | force] [-a conflicts] [-r untracked | ignored] [<file> ...]
clone <uri> [<path>]
commit [-a] [-m <message> | -F <filename>]
config user [-s global | local] [-n <username>] [-e <email>]
diff [-s] [-c]
fetch [<remote>]
init [-b] [-q] [-s true | false | group | all | world | umask] [-t <file>] [-n (no initial commit)] [<path>]
merge [-s safe | create | force]
pull
push [-t | <refspec>]
remote add <name> <uri>
remote show
rm [-b] [-s <stage>] <file>
status [-b] [-f short | long | porcelain]
tag -d <tag-name> | [-f] [-m <message>] <tag-name> [<commit> | <object>]
tag list [-l <number>] [pattern]

Repositories and your login directory must be on an ODS-5 file systems
Files must be stream-lf

This is a basic summary of the available commands. If you now want a few more details on a specific command, run the program specifying the command in question and supply some invalid option or leave out a required parameter. For example:

$ vgit2 init -h
init: illegal option -- h
Usage: init [-b] [-q] [-s true | false | group | all | world | umask] [-t <file>] [-n] [<path>]

Options:
-b Create a bare repository (has no working tree).
-q Quiet (only display errors and warnings).
-s <value> Repsoitory sharing (permitted values are "true", "false", "group",
"all", "world", and "umask")
-t <file> Specify a template directory.
-n Do not perform an initial commit.

Arguments:
<path> Directory where the repository is to be created (the directory is created
if necessary). If not specified, the current location is used.

You will notice that some commands map reasonably well to real git (although some options may not be available); other commands do not map quite so well (currently).
For the LLVM work John R mentioned, the compiler are using a primary repository on shared disk. Developers clone the repo into their private work areas and push changes back to the primary repo so that others can sync off that by doing a pull. My guess is that you will most likely want to use it with some service like GitHub, in which case typical usage will be pretty much exactly as per real git.

Some other points to note:

1. All files must be stream-lf. The program (currently) will exit with an error if you try to add a file to the repo that is not stream-lf. If for any reason the “add” command fails, note that nothing will have been added to the repository (it’s an all or nothing scenario), and you can safely fix up the problem (maybe you have to convert a file to stream-lf) and re-run the “add” command.

o When adding some number of files I would suggest using the “-v” (verbose) flag. This will allow you to see which file the add operation is failing on (probably because the file is not stream-lf).

2. ODS-5 only.

3. You may notice that it will be a little slow process to do an initial load or to clone repositories with a large number of files, but you only need to do this once, after which committing, pushing, and pulling changes should not be too painful.

4. If you want to use key-based authentication (as opposed to username/password over HTTPS, for example), you will need to define the logical names git$id_rsa and git$id_rsa_pub to map to the private and public key files, respectively (and of course your public key will need to be registered with the service (GitHub or whatever).

I’m sure you will encounter things that may not work as expected, but rather than me trying to pre-empt every possible scenario, it’s probably better for you to give it a spin and email me if/when you have problems and/or would like additional functionality added.