Feature #105

an auto-push to bitbucket.org, etc, would be nice

Added by Rich Lowe over 3 years ago. Updated over 3 years ago.

Status:Closed Start date:2010-08-28
Priority:Low Due date:
Assignee:Bryan Horstmann-Allen % Done:

0%

Category:- Spent time: -
Target version:-
Difficulty:Medium Tags:needs-triage

Description

It'd be great to hook an automatic push to an illumos project on bitbucket.org, and/or similar sites.

This should be as easy as

  • sign up
  • create a password-less ssh keypair
  • cron 'hg push -R illumos-gate -e 'ssh -Ci <privatekey>' ssh:///<user>/<tree>

using cron, rather than a hook, avoids blocking clients on arbitrarily long push (and in the case of git, conversion) processing,
for git, it appears that 'fast-export' (http://repo.or.cz/w/fast-export.git) is the tool of choice, and I assume github and gitorious
the preferred destinations.

History

Updated by Andre van Eyssen over 3 years ago

  • Assignee set to Andre van Eyssen
  • Priority changed from Normal to Low

Ack'd.

Updated by Andre van Eyssen over 3 years ago

  • Assignee changed from Andre van Eyssen to Bryan Horstmann-Allen

Assigning to bdha, our latest most dedicated victim. Err, volunteer.

Updated by Rich Lowe over 3 years ago

bitbucket is easy, since both ends are Hg, just requires an account and a cron job.

git is less easy, fast-export, as best as I know, is uni-directional. There's also hg-git, from the github folks which is bi-directional. Unfortunately, the other major difference between the two is that hg-git, at least for the initial push, is not fast (it has been running for 24 hours now, in my attempt at trying it out). Obviously incremental performance is more important, but I haven't got far enough to get to try yet.

I registered the 'illumos' organization on github a couple of days ago, figuring I'd take this bug and get an interim solution going. If folks (bdha?) need to be added to it somehow, let me know (email or IRC).

Updated by Jeppe Toustrup over 3 years ago

I managed to get illumos-gate pushed to github: http://github.com/Tenzer/illumos-gate. It took around 16-17 hours to convert all the mercurial data to git, and then hg-git failed with an "out of memory" error, probably because the memory limit for one process on my 32 bit box was reached.
Since the data converted to git was placed in a bare repository in illumos-gate/.hg/git/, it was simple to just do a git push to github from there.

Two commits came in during the conversion process, so I had the chance to check how incremental updates worked, and it only takes the new changesets and converts, so it's a fairly quick process. I did however had to do a "git push" again from the before mentioned directory, since hg-git doesn't seem to be able to handle such an amount of data on my system.

I don't know if the project I have under my account on github can be moved to the illumos organization, otherwise I don't mind uploading it again, since I already have the data converted from hg to git.

Also, I could set up a cronjob to do a push once in a while when new changes have been committed, if need be.

Updated by Bryan Horstmann-Allen over 3 years ago

Pulled Jeppe's tarball to w02:/export/scratch.

Updated by Jeppe Toustrup over 3 years ago

The steps the cronjob should do is as follows:

[enter illumos-gate hg directory]
hg pull -u
hg gexport
cd .hg/git/
git push

It shouldn't matter if there are no new commits when running this, since all the commands have light operations in case there are no new changes.

The steps required to be set up on the cronjob server before doing this, would be to install hg, git and hg-git. Follow the instructions on how to configure hg-git, and then create a password-less SSH key which will be used to push to GitHub.

Updated by Bryan Horstmann-Allen over 3 years ago

hg-git and friends built. untar'd ball to /export/scratch/.hg. Just need to set up GH and start pushing.

Updated by Bryan Horstmann-Allen over 3 years ago

Now syncing hg to the github repo every 5 minutes.

http://github.com/illumos/illumos-gate

Updated by Bryan Horstmann-Allen over 3 years ago

  • Status changed from New to Closed

This process needs to be moved to a proper illumos host, but for now it works. When I get my little virtual network up, I'll Puppetize and move it there. Closing for now.

Also available in: Atom PDF