The main Perdigao data system configuration is in the public-field-projects repository on github.com: https://github.com/ncareol/public-field-projects.  Despite the name, this repository is no longer public, so your github account must have been given permission to access this repository.

This configuration is deployed to many places.  It is used to run the data processing on ustar and at EOL on barolo, besides existing on every single DSM in perdigao.  So it is important use revision control to keep the configuration consistent and to track all the changes.

This presents some challenges using git on the shared daq and isfs accounts.  The basic tips are available on this software engineering wiki page:

http://wiki.eol.ucar.edu/sew/GitSharedAccounts

The quick reference section provides enough to get started, but there are lots of details there in case of questions.  After logging in with the sshgit script posted on that page, you should be able to commit in <home>/isfs/projects, and you will be recorded as the git author.  Of course, this assumes that you are set as the git author on the host from which you're logging in.  In summary:

From a personal account:

  1. Visit the wiki page above and download the sshgit script, and set it executable.

  2. Make sure git config user.name and git config user.email are correct, otherwise set them like so:

    git config --global user.name "First Last"
git config --global user.email "username@ucar.edu"

  3. Make sure your current session can authenticate to github:

    git ls-remote git@github.com:/ncareol/public-field-projects

      If not, follow the steps on github.com to create a SSH key and authorize it for your github.com account.

  4. Log into the remote shared account with the script:

    sshgit daq@ops-perdigao.dyndns.org

From the shared account

Use git as usual on the shared account:

  1. cd isfs/projects/Perdigao
  2. git pull
  3. git commit
  4. After committing, it's a good idea to check the author information on the commit, just in case, with git show.
  5. git push

The same steps work for logging into the isfs account on barolo.  However, for that account, the operational configuration under ~isfs/isfs has group write permissions, so EOL users should be able to pull into that repository while logged into their own account.  You can also edit, commit, and push, but we risk interfering if multiple people edit there.  Same goes for /net/isf/isfs/projects.

Initial project creation:

  1. cd isfs/projects
  2. git add RELAMPAGO
  3. git commit RELAMPAGO
  4. cd RELAMPAGO
  5. git push

Fixing Authorship

If you forget to use the sshgit script to log in, you may still be able to commit to a repository, but the author of the commit will be wrong.  You can fix it like so:

First log off the remote host.
Log back in with sshgit.
cd to the repository
Fix the last commit: git commit --amend --reset-author

The git command will also prompt for a new commit message, but the previous commit message is provided as a default.

 

  • No labels

3 Comments

  1. npotts

    Gary,

    Rather than have this info here alone, it might be useful to post this same info as "README.md" or similar to the repo exactly.  

    I have to admit, I am rather confused on the sshgit part - I am not sure what you are trying to accomplish. If you are trying to keep track of authors editing the config, this seems like a painful way to do what is needed - in my mind, whatever is run on the remote systems should be using a single shared read-only deploy key that allows the remote nodes to pull the config from github, and by deffinition, are unable to push changes.  From there, anyone altering the config should have a local copy of the repo they edit and push whereas the edge nodes just do  a 'git checkout --depth=1 -f" and overwrite any changes

  2. Gary Granger

    I hadn't thought of including this in the repo, but since this applies to any repo I thought it should be more generally available.  I use public-field-projects as the example because that's where all the collaboration (and possible conflict) is happening at the moment.

    The point about using a deploy key is well taken, and certainly I encourage developing and testing on private repositories and then deploying to the production repository.  However, in practice that's going to take some time to make it feasible; some things are hard to test except on the production system.  We'll keep working on making the configuration more self-contained and easier to test standalone before deploying, but we're not ready for that yet.

    Finally, I didn't know about the --depth option to clone, thanks for the pointer.  That could be particularly handy on the embedded systems, if we were to use git as a way to track local changes to the configuration and also commit them back to the master repository.  That's one of the cases where we would want a simple mechanism to track authorship and allow pushes from a deployed configuration.  Sometimes the configuration has to be changed in the field directly on the DSM, while offline.

    Anyway, thanks for making the point about "one-way" deployments.  I should have made that point in the documentation.  The point isn't to advocate for kludges like this to propagate authorship and allow pushing from deployed configurations, the point is to facilitate using git to track configuration changes until we transition to something better.  (Erik would tell me that shared accounts are bad too, and though that's a good point, on field systems I don't have a clean solution for that yet either.)

     

    1. npotts

      At this point I am just figuring that you all are just trying to get by, so some of the ideals are going to get sacrificed, and I know I am missing some of the particular work scenarios.  For future reference, it might be worth coming up with a schema or methodology for use in the future.