Anonymous DVCS in the Darknet.
This is a mirror of the documentation of the infocalypse extension for Mercurial written by djk - published here with his permission. It is licensed solely under the GPLv2 or later.
Your code is then hosted decentrally and anonymously, making it just as censorship-resistant as all other content in Freenet.
It works better than the other DVCS currently available for Freenet.
Most of the information you will find in this document can also be found in the extension's online help. i.e.:
hg help infocalypse
The extension has the following dependencies:
You probably won't have to worry about installing Python. It's included in the Windows binary Mercurial distributions and most *nix flavor OS's should have a reasonably up to date version of Python installed.
Version 1.0.2 won't work.
I use version 1.2.1 (x86 Gentoo) on a daily basis. Later versions should work.
I've smoke tested 1.1.2 (on Ubuntu Jaunty Jackalope) and 1.3 (on Widows XP) without finding any problems.
The official FMS freesite is here:
Here are step-by-step instructions on how to install the extension.
You'll get a Potentially Dangerous Content warning from fproxy because the mime type isn't set. Choose 'Click here to force your browser to download the file to disk.'.
I'll refer to the directory that you saved the bundle file to as DOWNLOAD_DIR.
cd INSTALL_DIR hg init
hg pull DOWNLOAD_DIR/infocalypse2_1723a8de6e7c.hg hg update
# .hgrc/mercurial.ini snippet [extensions] infocalypse = INSTALL_DIR/infocalypse
where INSTALL_DIR is the directory you unbundled into.
If you don't known where to find/create your .hgrc/mercurial.ini file this link may be useful:
http://www.selenic.com/mercurial/hgrc.5.html [HTTP Link!]
If you run your Freenet node on another machine or on a non-standard port you'll need to use the --fcphost and/or --fcpport parameters to set the FCP host and port respectively.
By default fn-setup will write the configuration file for the extension (.infocalype on *nix, infocalypse.ini on Windows) into your home directory and also create a temp directory called infocalypse_tmp there.
You can change the location of the temp directory by using the --tmpdir argument.
If you want to put the config file in a different location set the cfg_file option in the [infocalypse] section of your .hgrc/mercurial.ini file before running fn-setup.
Example .hgrc entry: # Snip, from .hgrc [infocalypse] cfg_file = /mnt/usbkey/s3kr1t/infocalypse.cfg
# Example .infocalypse snippet fms_id = YOUR_FMS_ID fms_host = 127.0.0.1 fms_port = 1119
where YOUR_FMS_ID is the part of your fms id before the '@' sign.
If you run FMS with the default settings on the same machine you are running
Mercurial on you probably won't need to adjust the fcp_host or fcp_port.
You can skip this step if you're not running fms.
hg fn-fmsread -v
You can skip this step if you're not running fms.
hg fn-pull --aggressive --debug --uri USK@kRM~jJVREwnN2qnA8R0Vt8HmpfRzBZ0j4rHC2cQ-0hw,2xcoQVdQLyqfTpF2DpkdUIbHFCeL4W~2X1phUYymnhM,AQACAAE/infocalypse.hgext.R1/41 hg update
You may have trouble finding the top key if you're not using fn-fmsread. Just keep retrying. If you know the index has increased, use the new index in the URI.
After the first pull, you can update without the URI.
This extension is under active development. You should periodically update to get the latest bug fixes and new features.
hg fn-fmsread -vhg fn-pull --aggressive hg update
If you're not running FMS you can skip the fn-fmsread step. You may have trouble getting the top key. Just keep retrying.
If you're having trouble updating and you know the index has increased, use the full URI with the new index as above.
Here's background information that's useful when using the extension.See the
Infocalypse 2.0 hg extension page on my freesite for a more detailed description of how the extension works.
An Infocalypse repository is just a collection of hg bundle files which have been inserted into Freenet as CHKs and some metadata describing how to pull the bundles to reconstruct the repository that they represent. When you 'push' to an infocalypse repository a new bundleCHK is inserted with the changes since the last update. When you 'pull', only the CHKs for bundles for changesets not already in the local repository need to be fetched.
The latest version of the repository's metadata is stored on a Freenet Updateable Subspace Key (USK) as a small binary file.
You'll notice that repository USKs end with a number without a trailing '/'. This is an important distinction. A repository USK is not a freesite. If you try to view one with fproxy you'll just get a 'Potentially Dangerous Content' warning. This is harmless, and ugly but unavoidable at the current time because of limitation in fproxy/FCP.
Repository USKs that end in *.R1/<number> are inserted redundantly, with a second USK insert done on *.R0/<number>. Top key redundancy makes it easier for other people to fetch your repository.
Inserting to a redundant repository USK makes the inserter more vulnerable to
correlation attacks. Don't use '.R1' USKs if you're worried about this.
Repository USKs can be long and cumbersome. A repository hash is the first 12 bytes of the Sha1 hash of the zero index version of a repository USK. e.g.:
SHA1( USK@kRM~jJVREwnN2qnA8R0Vt8HmpfRzBZ0j4rHC2cQ-0hw,2xcoQVdQLyqfTpF2DpkdUIbHFCeL4W~2X1phUYymnhM,AQACAAE/infocalypse.hgext.R1/0 ) == 'be68e8feccdd'
You can get the repository hash for a repository USK using:
from a directory the repository USK has been fn-pull'd into.
You can get the hashes of repositories that other people have announced via fms with:
hg fn-fmsread --listall
Repository hashes are used in the fms update trust map.
When you run fn-setup, it creates a default SSK private key, which it stores inthe default_private_key parameter in your .infocalypse/infocalypse.ini file.
You can edit the config file to substitute any valid SSK private key you want.
If you specify an Insert URI without the key part for an infocalypse command the default private key is filled in for you. i.e
hg fn-create --uri USK@/test.R1/0
Inserts the local hg repository into a new USK in Freenet, using the private key in your config file.
The extension's commands 'remember' the insert and request repository USKs they were last run with when run again from the same directory.
This makes it unnecessary to retype cumbersome repository USK values once a repository has been successfully pulled or pushed from a directory.
fn-pull and fn-push have an --aggressive command line argument which causes them to search harder for the latest request URI.
This can be slow, especially if the USK index is much lower than the latest index in Freenet.
You will need to use it if you're not using FMS update notifications.
Here are examples of basic commands.
You can generate an new private key with:
This has no effect on the stored default private key.
Make sure to change the 'SSK' in the InsertURI to 'USK' when supplying the insert URI on the command line.
hg fn-create --uri USK@/test.R1/0
Inserts the local hg repository into a new USK in Freenet, using the privatekey in your config file. You can use a full insert URI value if you want.
If you see an "update -- Bundle too big to salt!" warning message when you run this command you should consider running
fn-reinsert --level 4.
hg fn-push --uri USK@/test.R1/0
Pushes incremental changes from the local directory into an existing Infocalypse repository.
The <keypart>/test.R1/0 repository must already exist in Freenet.In the example above the default private key is used. You could have specified a full Insert URI. The URI must end in a number but the value doesn't matter because fn-push searches for the latest unused index.
You can ommit the --uri argument whenyou run from the same directory the fn-create (or a previous fn-push)was run from.
hg fn-pull --uri <request uri>
pulls from an Infocalypse repository in Freenet intothe local repository.
Here's an example with a fully specified uri.
You can ommit the --uri argument whenyou run from the same directory a previous fn-pull was successfully run from.
For maximum reliability use the --aggressive argument.
The extension can send and receive repository update notifications via FMS. It is highly recommended that you
setup this feature.
There's a trust map in the .infocalypse/infocalypse.ini config file which determines which fms ids can update the index values for which repositories. It is purely local and completely separate from the trust values which appear in the FMS web of trust.
The format is:
<number> = <fms_id>|<usk_hash0>|<usk_hash1>| ... |<usk_hashn>
The number value must be unique, but is ignored.
The fms_id values are the full FMS ids that you are trusting to update the repositories with the listed hashes.
The usk_hash* values are repository hashes.
Here's an example trust map config entry:
# Example .infocalypse snippet [fmsread_trust_map] 1 = test0@adnT6a9yUSEWe5p8J-O1i8rJCDPqccY~dVvAmtMuC9Q|55833b3e6419 0 = djk@isFiaD04zgAgnrEC5XJt1i4IE7AkNPqhBG5bONi6Yks|be68e8feccdd|5582404a9124 2 = test1@SH1BCHw-47oD9~B56SkijxfE35M9XUvqXLX1aYyZNyA|fab7c8bd2fc3
You must update the trust map to enable index updating for repos other than the one this code lives in (be68e8feccdd). You can edit the config file directly if you want.
However, the easiest way to update the trust map is by using the--trust and --untrust options on fn-fmsread.
For example to trust falafel@IxVqeqM0LyYdTmYAf5z49SJZUxr7NtQkOqVYG0hvITwto notify you about changes to the repository with repo hash 2220b02cf7ee,type:
hg fn-fmsread --trust --hash 2220b02cf7ee --fmsid falafel@IxVqeqM0LyYdTmYAf5z49SJZUxr7NtQkOqVYG0hvITw
And to stop trusting that FMS id for updates to 2220b02cf7ee, you would type:
hg fn-fmsread --untrust --hash 2220b02cf7ee --fmsid falafel@IxVqeqM0LyYdTmYAf5z49SJZUxr7NtQkOqVYG0hvITw
To show the trust map type:
hg fn-fmsread --showtrust
hg fn-fmsread -v
Will read update notifications for all the repos in the trust map and locally cache the new latest index values. If you run with -vit prints a message when updates are available which weren't used because the sender(s) weren't in the trust map.
hg fn-fmsread --list
Displays announced repositories from fms ids that appear inthe trust map.
hg fn-fmsread --listall
Displays all announced repositories including ones from unknown fms ids.
You can use the --hash option with fn-pull to pull any repository you see in the fn-read --list or fn-read --listall lists.
For example to pull the latest version of the infocalypse extension code, cd to an empty directory and type:
hg inithg fn-pull --hash be68e8feccdd --aggressive
hg fn-fmsnotify -v
Posts an update notification for the current repository to fms.
You MUST set the fms_id value in the config fileto your fms id for this to work.
Use --dryrun to double check before sending the actual fms message.
Use --announce at least once if you want your USK to show up in the fmsread --listall list.
By default notifications are written to and read from the infocalypse.notify fms group.
The read and write groups can be changed by editing the following variables in the config file:
fmsnotify_group = <group>
fmsread_groups = <group0>[|<group1>|...]
fms can have pretty high latency. Be patient. It may take hours (sometimes a day!) for your notification to appear. Don't send lots of redundant notifications.
will re-insert the bundles for the repository that was last pulled into the directory.
The exact behavior is determined by the level argument.
This is the default level.
Levels 1 and 4 require that you have the privatekey for the repository. For other levels, the top key insert is skipped if you don't have the private key.
DO NOT use fn-reinsert if you're concerned about
correlation attacks. The risk is on the order of re-inserting a freesite, but may be worse if you use redundant(i.e. USK@<line noise>/name.R1/0) top keys.
hg fn-copy --inserturi USK@/name_for_my_copy.R1/0
copies the Infocalypse repository which was fn-pull'd intothe local directory onto a new repository USK under your default private key. You can use a full insert URI if you want.
This only requires copying the top key data (a maximum of 2 SSK inserts).
It is possible for multiple people to collaborate anonymously over Freenet by sharing the private key to a single Infocalypse repository.
The FreeFAQ is an example of this technique.
Here are some things to keep in mind when sharing private keys.
USK@<public_key>/revoked.txt, I stop using the key.
hg fn-putsite --index <n>
inserts a freesite based on the configuration inthe freesite.cfg file in the root of the repository.
hg fn-putsite --createconfig
to create a basic freesite.cfg file that you can modify. Look at the comments in it for an explanation of the supported parameters.
The default freesite.cfg file inserts using the same private key as the repo and a site name of 'default'. Editing the name is highly recommended.
You can use --key CHK@ to insert a test version of the site to a CHK key before writing to the USK.
I don't believe that using this extension is significantly more dangerous that using any other piece of Freenet client code, but here is a list of the risks which come to mind:
While written in Java, Freenet loads native code via JNI (FEC codecs, bigint stuff, wrapper, etc.) that makes it vulnerable to the same kinds of attacks as any other C/C++ code.
I personally build FMS from source and run it in a chroot jail.
Somedude, the author of FMS, seems like a reputable guy and has conducted himself as such for more than a year.
Any system which has this property is vulnerable. e.g. fproxy Freesite insertion,Freetalk, FMS, FLIP. This extension's optional use of
redundant top keys may make it particularly vulnerable. If you are concerned don't use '.R1' keys.
Running your node in pure darknet mode with trusted peers may somewhat reduce the risk of correlation attacks.
There are lots of eyes over the Mercurial and Python source.
You only need to insert/retrieve what has actually changed. Changes of up to 32kof compressed deltas can be fetched in as little as one SSK fetch and one CHK fetch.
The top level SSK and the CHK with the representation of the repository state are inserted redundantly so there are no 'critical path' keys. Updates of up to ~= 7Mbare inserted redundantly by cloning the splitfile metadata at the cost of a single32k CHK insert.
Anyone can re-insert all repository data except for the top level SSKs with a simple command (hg fn-reinsert). The repository owner can re-insert the top levelSSKs as well.
Older changes are automatically 'rolled up' into large splitfiles, such that the entire repository can almost always be fetched in 4 CHK fetches or less.
The authoritative repository for the extension's code is hosted in Freenet:
hg inithg fn-fmsread -vhg fn-pull --aggressive --debug --uri USK@kRM~jJVREwnN2qnA8R0Vt8HmpfRzBZ0j4rHC2cQ-0hw,2xcoQVdQLyqfTpF2DpkdUIbHFCeL4W~2X1phUYymnhM,AQACAAE/infocalypse.hgext.R1/41hg update
It is also mirrored on
This document is meant to inserted into Freenet.
It contains links (starting with '
CHK@' and '
USK@')to Freenet keys that will only work from within fproxy [HTTP link!].
You can find reasonably up to date version of this document on my freesite:
I lurk on the freenet and fms boards.
If you really need to you can email me at d kar bott at com cast dot net but I prefer FMS.