This is a brief guide for distributed development between anonymous and non-anonymous participants over freenet. I try to mirror all required content here.
This guide shows how to use infocalypse to contribute to freenet repositories anonymously. Also it shows how to get these changes back into the git repos on github.
For hosting in freenet it uses the very efficient infocalypse extension of Mercurial. Infocalypse supports incremental updates, additional redundancy, reinserting other peoples repositories without being able to tamper with them, minimizing the number of requests needed to get a repo and a whole lot of other nice features.
Since infocalypse works for Mercurial and most freenet development is done in git (and on github), we use hg-git to transform the repository from git to Mecurial and the other way round (using the dulwich library which implements the git protocol in Python). That bridge is transparent, so no history gets lost when pushing the data to freenet and back.
Anonymous coders can then use infocalypse to retrieve the repositories from freenet, hack on them and upload their new versions directly into freenet. Afterwards non-anonymous hackers can retrieve these changes and push them back into github.
These will be out of date, soon. But they should suffice for in-freenet development for a long time.
apt-get install python or emerge python, ….apt-get install mercurial or emerge mercurial, …This will use the commandline on GNU/Linux. If you want to provide instructions for GUI users, feel free to fork this repo.
hg init infocalypsecd infocalypsehg pull -u path/to/infocalypse.bundleecho >> ~/.hgrcecho "[extensions]" >> ~/.hgrcecho "infocalypse = " $(pwd)/infocalypse >> ~/.hgrchg fn-setuphg fn-pull --agressive --uri USK@6~ZDYdvAgMoUfG6M5Kwi7SQqyS-gTcyFeaNN1Pf3FvY,OSOT4OEeg4xyYnwcGECZUX6~lnmYrZsz05Km7G7bvOQ,AQACAAE/infocalypse.R1/0Done. You just added freenet capabilities to your DVCS and updated the repo over freenet.
To contribute to a project anonymously, you need someone who uploads the repository into freenet (lets call him core devel) and someone who can get your changes back to the core devs, for example by pushing them onto github.
So we can break up contributing into 3 tasks:
As an example we will do that with the pyFreenet repo (since inserting and downloading the fred-repo takes some time).
Shown with the example of pyFreenet.
To mirror a repo from github to freenet, you need hg-git and dulwich.
For installing and activating hg-git, let’s just ask the hg-git website:
First, make sure that you have a working C compiler on your system. On Windows, one option is the free Microsoft Visual C++. (Be sure to install the 2008 version for compatibility reasons.) On Debian-style Linux, run apt-get install python-dev.
Secondly, run easy_install hg-git. If you don't have easy_install available, you can get it as part of Python's setuptools package: apt-get install python-setuptools.
Lastly, make sure the following is in your ~/.hgrc:
[extensions]
hgext.bookmarks =
hggit =
Now you can simply clone the repository
hg clone git+https://github.com/freenet/pyFreenetcd pyFreenethg clone . freenet:USK@/pyFreenethg clone gives you a REQUEST_KEY at the end. For my pyFreenet upload it gave me this:
USK@-sEkUkv7SClGNhjpvFF9TtmGHnoiwvThkepMUTy7Fh4,Rciq6x0804hb~rNzxmINfkuSCgAEvHZjJNHQcUo0nuQ,AQACAAE/pyfreenet.R1/0
Share the request key to enable others to get the repo.
Never share the insert key, except if you want to pass on maintenance of your in-freenet repo.
To get a repo, we first initialize an empty repo and then use infocalypse to pull the changes from the given REQUEST_KEY:
hg init REPOcd REPOhg fn-pull --aggressive --uri USK@-sEkUkv7SClGNhjpvFF9TtmGHnoiwvThkepMUTy7Fh4,Rciq6x0804hb~rNzxmINfkuSCgAEvHZjJNHQcUo0nuQ,AQACAAE/pyfreenet.R1/0(And wait a bit. For me this call on a test repo took 3 and a half minutes)
--aggressive tells infocalypse to search harder for the most current USK revision.
Then we hack on the repo (if you are more comfortable with git than with Mercurial, you can use hg-git to push it to a git repo and pull back the changes once you are done).
To share our work, we push the code back into freenet:
hg fn-genkeyhg fn-create --uri [the generated insert key] # replace NAME with REPONAME.R1This gives us ANOTHER_REQUEST_KEY.
Now share the request key.
Never share the insert key, except if you want to pass on maintenance of your in-freenet repo.
If you want to get a notice when a repo gets updated, just bookmark it like any other freesite and fred will keep you up to date.
To get the changes back onto github, we simply do the reverse path of getting the repo from github: Pull from freenet, then push to github. But this time we use an authenticated ssh connection for interacting with github.
So essentially, you need to contact a core dev and ask him to do the following:
hg fn-pull --aggressive --uri ANOTHER_REQUEST_KEYhg push git+ssh://git@github.com/USER/pyfreenet.gitIf you want to hack directly on fred, you can get it with the following key:
hg pull USK@yJUguKTfUHgVutplApc8A3ucq~QogPfqx3-1ZunKjYk,EzZzXErTnhC~ll7HGpgDDik15KTFlwdpuGcRA7HL5uk,AQACAAE/fred.R1/1
It is uploaded once a week. Nag ArneBab on Sone if something breaks :)
(some repetition to come)
hg fn-genkeyhg fn-create --uri [the generated insert key] # replace NAME with REPONAME.R1hg init REPOcd REPOhg fn-pull --aggressive --uri REQUEST_KEYhg fn-genkeyhg fn-create --uri [the generated insert key] # replace NAME with REPONAME.R1Included here, because github is currently used as non-anonymous development platform.
Enable the hg-git extension.
hg clone git+https://github.com/USER/repo.gitcd repohg fn-genkeyhg fn-create --uri [the generated insert key] # replace NAME with REPONAME.R1To get the changes back onto github, do the reverse:
hg fn-pull --aggressive --uri SOME_REQUEST_KEYhg push git+ssh://git@github.com/USER/repo.githg fn-info
Content in freenet can fall out if it is not requested. These are tipps to make sure that repos you consider important stay available.
Reinsert it. There are different levels of reinsertion. 1 and 4 can only be done by the creator, all the others can be done by anyone who downloaded the repositories. The levels:
level: 1 - re-inserts the top key(s) 2 - re-inserts the top keys(s), graphs(s) and the most recent update. 3 - re-inserts the top keys(s), graphs(s) and all keys required to bootstrap the repo. This is the default level. 4 - adds redundancy for big (>7Mb) updates. 5 - re-inserts existing redundant big updates.
Now reinsert, since this is our own repo, we use all levels:
for i in 1 2 3 4 5; do hg fn-reinsert --level $i; doneGet it in a new repo from time to time to ensure that it is still available:
cd REPOhg fn-info # get the REQUEST_KEYhg init TMPREPOcd TMPREPOhg fn-pull --aggressive --uri REQUEST_KEYIf that should not work, just use hg fn-create --uri INSERT_URI again (with the key from hg fn-info.
Reinsert all parts for which you do not need the insert key:
for i in 2 3 5; do hg fn-reinsert --level $i; donehg help infocalypse
If you wish to contribute, just clone and fork this repo.
Request key: USK@ugGhE7lfI-QyFYL6yuZt7uyhlrVfvIRo0xkQxd2qV~c,L1bCwDm5Hh5qgqml7SBQcF85vL8IPvzN4WihjWIKH8I,AQACAAE/infocalypse-dvcs-over-freenet.R1/1
The not-just-downloaded files (and dulwich) are in static/
Currently the site gets parsed by pymarkdown_minisite, which is not yet included in this repo (⇒TODO: Add a small parser). Since it is not yet included, you can either edit the index.txt (preferred) or directly work on the static/index.html to add something to the site.
Then just upload the static/ folder via freesitemgr (from pyFreenet) or jSite - and share the code via hg fn-create --uri INSERT_URI.
Infocalypse: code sharing over Freenet-- 2012-04-22 08:47:55 --
Created with pyMarkdown Minisite
using the layout from the pybrary.