Light

warning: Creating default object from empty value in /www/htdocs/w0062791/arnebab/modules/taxonomy/taxonomy.pages.inc on line 33.

Tail Call Optimization (TCO), dependency, broken debug builds in C and C++ — and gcc 4.8

TCO: Reducing the algorithmic complexity of recursion.
Debug build: Add overhead to a program to trace errors.
Debug without TCO: Obliterate any possibility of fixing recursion bugs.

“Never develop with optimizations which the debug mode of the compiler of the future maintainer of your code does not use.”°

UPDATE: GCC 4.8 gives us -Og -foptimize-sibling-calls which generates nice-backtraces, and I had a few quite embarrassing errors in my C - thanks to AKF for the catch!

1 Intro

Tail Call Optimization (TCO) makes this

def foo(n):
    print(n)
    return foo(n+1)
foo(1)

behave like this

def foo(n):
    print(n)
    return n+1
n = 1 while True: n = foo(n)

Test of the hg evolve extension for easier upstreaming

1 Rationale

PDF-version (for printing)

orgmode-version (for editing)

repository (for forking)

Currently I rework my code extensively before I push it into upstream SVN. Some of that is inconvenient and it would be nicer to have easy to use refactoring tools.

hg evolve might offer that.

This test uses the mutable-hg extension in revision c70a1091e0d8 (24 changesets after 2.1.0). It will likely be obsolete, soon, since mutable-hg is currently moved into Mercurial core by Pierre-Yves David, its main developer. I hope it will be useful for you, to assess the future possibilities of Mercurial today. This is not (only) a pun on “obsolete”, the functionality at the core of evolve which allows safe, collaborative history rewriting ☺

The dynamics of free culture and the danger of noncommercial clauses

NC covered works trick people into investing in a dead end

Free licensing lowers the barrier of entry to creating cultural works, which unlocks a dynamic where people can realize their ideas much easier - and where culture can actually live, creating memes, adjusting them to new situations and using new approaches with old topics.

But for that to really take off, people have to be able to make a living from their creations - which build on other works.

Motivation and Reward

Debunking the myth that you can increase the performance of creative workers with carrot and stick.

Update: I sent this text to the gnu maintainers, and after the original article had been offline for several years, they now managed to convince Alfie Kohn to allow them to distribute the article again. So Studies Find Reward Often No Motivator is finally online again! → gnu.org/p/motivation.html
My text might not have been included on the GNU websites, but it fullfilled its purpose - though in a different way than I had expected.

Update: I got the feedback that some messages in this article are still unclear. It should not implicate, that “in order to increase motivation in the free software world people need to be offered a high income and a long term contract”. Paying a good income in a long term contract is a way to avoid the harmful effect payment can have on performance while enabling someone to work full-time on the project. An empirical study found, that the source and intensity of motivation of free software developers does not differ significantly between people who work for hire and people who work without payment, so many companies employing free software developers seem to do it right (or only the companies who do it right can keep their free software programmers).1

A few months ago, the GNU project had to withdraw its article on motivation and monetary reward, because its author did not allow them to spread it anymore. So I recreated its core - with references to solid research.

Executive Summary

For creative tasks, the quality of performance strongly correllates with intrinsic motivation: Being interested in the task itself.

This article will only talk about that.

The main factors which are commonly associated with intrinsic motivation are:

  • Positive verbal feedback which increases intrinsic motivation.
  • Payment independent of performance which actually has no effect.
  • Payment dependent on performance which reduces the motivation on the long term.
  • Negative verbal feedback which directly reduces intrinsic motivation.
  • Threatening someone with punishment which strongly reduces intrinsic motivation.

To make it short: Anything which diverts the focus from the task at hand towards some external matter (either positive or negative) reduces the intrinsic motivation and that in turn reduces work performance.

If you want to help people perform well, make sure that they don’t have to worry about other stuff besides their work and give them positive verbal feedback about the work they do.

Note: In the paper »Why Hackers Do What They Do: Understanding Motivation and Effort in Free/Open Source Software Projects« from 2005, Karim R. Lakhani and Robert G Wolf showed empirically that the payment people get to work in free software projects has no detrimental effect on their intrinsic motivation. In their sample 40% of the developers were paid for their work on free software projects and their intrinsic motivation was as high as the motivation of unpaid developers.


  1. We find […], that enjoyment-based intrinsic motivation, namely how creative a person feels when working on the project, is the strongest and most pervasive driver. The source and intensity of motivation of free software developers does not differ significantly between people who work for hire and people who work without payment. From Why Hackers Do What They Do: Understanding Motivation and Effort in Free/Open Source Software Projects by Karim R. Lakhani* and Robert G Wolf** from the * MIT Sloan School of Management | The Boston Consulting Group and ** The Boston Consulting Group. 

Neither Humble nor Indie Bundle

Comment to New Humble Bundle Is Windows Only, DRM Games.

The new Humble Indie Bundle is no longer free, indie, cross-plattform or user-respecting.

How to make companies act ethically

→ comment on Slashdot concerning Unexpected methods to promote freedom?

Was it really Apple who ended DRM? Would they have done so without the protests and evangelizing against DRM? Without protesters in front of Apple Stores? And without the many people telling their friends to just not accept DRM?

That “preaching” created a situation where Apple could reap monetary gain from doing the right thing.

Insert a scaled screenshot in emacs org-mode

@marjoleink asked on identi.ca1, if it is possible to use emacs org-mode for showing scaled screenshots inline while writing. Since I thought I’d enjoy some hacking, I decided to take the challenge.

It does not do auto-scaling of embedded images, as far as I know, but the use case of screenshots can be done with a simple function (add this to your ~/.emacs or ~/.emacs.d/init.el):


  1. Matthew Gregg: @marjoleink "way of life" thing again, but if you can invest some time, org-mode is a really powerful note keeping environment. → Marjolein Katsma: @mcg I'm sure it is - but seriously: can you embed a diagram2 or screenshot, scale it, and link it to itself? 

  2. For diagrams, you can just insert a link to the image file without description, then org-mode can show it inline. To get an even nicer user-experience (plain text diagrams or ascii-art), you can use inline code via org-babel using graphviz (dot) or ditaa - the latter is used for the diagrams in my complete Mercurial branching strategy

Sending email to many people with Emacs Wanderlust

I recently needed to send an email to many people1.

Putting all of them into the BCC field did not work (mail rejected by provider) and when I split it into 2 emails, many did not see my mail because it was flagged as potential spam (they were not in the To-Field)2.

I did not want to put them all into the To-Field, because that would have spread their email-addresses around, which many would not want3.

So I needed a different solution. Which I found in the extensibility of emacs and wanderlust4. It now carries the name wl-draft-send-to-multiple-receivers-from-buffer.

You simply write the email as usual via wl-draft, then put all email addresses you want write to into a buffer and call M-x wl-draft-send-to-multiple-receivers-from-buffer. It asks you about the buffer with email addresses, then shows you all addresses and asks for confirmation.

Then it sends one email after the other, with a randomized wait of 0-10 seconds between messages to avoid flagging as spam.

If you want to use it, just add the following to your .emacs:

(defun wl-draft-clean-mail-address (address)
  (replace-regexp-in-string "," "" address))
(defun wl-draft-send-to-multiple-receivers (addresses) (loop for address in addresses do (progn (wl-user-agent-insert-header "To" (wl-draft-clean-mail-address address)) (let ((wl-interactive-send nil)) (wl-draft-send)) (sleep-for (random 10)))))
(defun wl-draft-send-to-multiple-receivers-from-buffer (&optional addresses-buffer-name) "Send a mail to multiple recipients - one recipient at a time" (interactive "BBuffer with one address per line") (let ((addresses nil)) (with-current-buffer addresses-buffer-name (setq addresses (split-string (buffer-string) "\n"))) (if (y-or-n-p (concat "Send this mail to " (mapconcat 'identity addresses ", "))) (wl-draft-send-to-multiple-receivers addresses))))

Happy Hacking!


  1. The email was about the birth of my second child, and I wanted to inform all people I care about (of whom I have the email address), which amounted to 220 recipients. 

  2. Naturally this technique could be used for real spamming, but to be frank: People who send spam won’t need it. They will already have much more sophisticated methods. This little trick just reduces the inconvenience brought upon us by the measures which are necessary due to spam. Otherwise I could just send a mail with 1000 receivers in the BCC field - which is how it should be. 

  3. It only needs one careless friend, and your connections to others get tracked in facebook and the likes. For more information on Facebook, see Stallman about Facebook

  4. Sure, there are also template mails and all such, but learning to use these would consume just as much time as extending emacs - and would be much less flexible: Should I need other ways to transform my mails, I’ll be able to just reuse my code. 

Bootstrapping the Freenet WoT with GnuPG - and GnuPG with Freenet

Intro

When you enter the freenet Web of Trust, you first need to get some trust from people by solving captchas. And even when people trust you somehow, you have no way to prove your identity in an automatic way, so you can’t create identities which freenet can label as trusted without manual intervention from your side.

Proposal

To change this, we can use the Web of Trust used in GnuPG to infer trust relationships between freenet WoT IDs.

Practically that means:

  • Write a message: “I am the WoT ID USK@” (replace with the

Easily converting ris-citations to bibtex with emacs and bibutils

The problem

Nature only gives me ris-formatted citations, but I use bibtex.

Also ris is far from human readable.

The background

ris can be reformatted to bibtext, but doing that manually disturbs my workflow when getting references while taking note about a paper in emacs.

I tend to search online for references, often just using google scholar, so when I find a ris reference, the first data I get for the ris-citation is a link.

The solution

Emacs

Cross platform, Free Software, almost all features you can think of, graphical and in the shell: Learn once, use for everything. » Get Emacs «

Emacs is a self-documenting, extensible editor, a development environment and a platform for lisp-programs - for example programs to make programming easier, but also for todo-lists on steroids, reading email, posting to identi.ca, and a host of other stuff (learn lisp).

It is also one of the origins of GNU and free software (Emacs History).

In Markdown-mode it looks like this:

Emacs mit Markdown mode

A complete Mercurial branching strategy

This is a complete branching strategy for Mercurial with optional adaptions for maintaining multiple releases1. It shows you all the actions you may need to take, except for those already described in the guide Mercurial in workflows.

For examples it uses the command-line UI, but it can easily be used with graphical Mercurial interfaces like TortoiseHG.

A simpler workflow for groups who need no separate stable branch is described in Feature seperation via named branches.

Summary

Firstoff, any model to be used by people should consist of simple, consistent rules. Programming is complex enough without having to worry about elaborate branching directives. Therefore this model boils down to 3 simple rules:

(1) you do all the work on default2 - except for hotfixes.

(2) on stable you only do hotfixes, merges for release3 and tagging for release. Only maintainers4 touch stable.

(3) you can use arbitrary feature-branches5, as long as you don’t call them default or stable. They always start at default (since you do all the work on default).

Diagram

To visualize the structure, here’s a 3-tiered diagram. To the left are the actions of developers (commits and feature branches) and in the center the tasks for maintainers (release and hotfix). The users to the right just use the stable branch.6

Overview Diagram
An overview of the branching strategy. Click the image to get the emacs org-mode ditaa-source.

Table of Contents

Practial Actions

Now we can look at all the actions you will ever need to do in this model:7

  • Regular development

    • commit changes: (edit); hg ci -m "message"

    • continue development after a release: hg update; (edit); hg ci -m "message"

  • Feature Branches

    • start a larger feature: hg branch feature-x; (edit); hg ci -m "message"

    • continue with the feature: hg update feature-x; (edit); hg ci -m "message"

    • merge the feature: hg update default; hg merge feature-x; hg ci -m "merged feature x into default"

    • close and merge the feature when you are done: hg update feature-x; hg ci --close-branch -m "finished feature x"; hg update default; hg merge feature-x; hg ci -m "merged finished feature x into default"

  • Tasks for Maintainers

    • Initialize (only needed once)

      • create the repo: hg init reponame; cd reponame

      • first commit: (edit); hg ci -m "message"

      • create the stable branch and do the first release: hg branch stable; hg tag tagname; hg up default; hg merge stable; hg ci -m "merge stable into default: ready for more development"

    • apply a hotfix8: hg up stable; (edit); hg ci -m "message"; hg up default; hg merge stable; hg ci -m "merge stable into default: ready for more development"

    • do a release9: hg up stable; hg merge default; hg ci -m "merged default into stable for release" ; hg tag tagname; hg up default ; hg merge stable ; hg ci -m "merged stable into default: ready for more development"


  1. if you need to maintain multiple very different releases simultanously, see or 10 for adaptions 

  2. default is the default branch. That’s the named branch you use when you don’t explicitely set a branch. Its alias is the empty string, so if no branch is shown in the log (hg log), you’re on the default branch. Thanks to John for asking! 

  3. If you want to release the changes from default in smaller chunks, you can also graft specific changes into a release preparation branch and merge that instead of directly merging default into stable. This can be useful to get real-life testing of the distinct parts. For details see the extension Graft changes into micro-releases

  4. Maintainers are those who do releases, while they do a release. At any other time, they follow the same patterns as everyone else. If the release tasks seem a bit long, keep in mind that you only need them when you do the release. Their goal is to make regular development as easy as possible, so you can tell your non-releasing colleagues “just work on default and everything will be fine”. 

  5. This model does not use bookmarks, because they don’t offer benefits which outweight the cost of introducing another concept, and because named branches for feature branches offer the advantage, that new programmers never get the code from a feature-branch when they clone the repository. For local work and small features, bookmarks can be used quite well, though, and since this model does not define their use, it also does not limit it.
    Additionally bookmarks could be useful for feature branches, if you use many of them (in that case reusing names is a real danger and not just a rare annoyance, and if you have a recent Mercurial, you can use the @ bookmark to signify the entry point for new clones) or if you use release branches:
    “What are people working on right now?” → hg bookmarks
    “Which lines of development do we have in the project?” → hg branches 

  6. Those users who want external verification can restrict themselves to the tagged releases - potentially GPG signed by trusted 3rd-party reviewers. GPG signatures are treated like hotfixes: reviewers sign on stable (via hg sign without options) and merge into default. Signing directly on stable reduces the possibility of signing the wrong revision. 

  7. hg pull and hg push to transfer changes and hg merge when you have multiple heads on one branch are implied in the actions: you can use any kind of repository structure and synchronization scheme. The practical actions only assume that you synchronize your repositories with the other contributors at some point. 

  8. Here a hotfix is defined as a fix which must be applied quickly out-of-order, for example to fix a security hole. It prompts a bugfix-release which only contains already stable and tested changes plus the hotfix. 

  9. If your project needs a certain release preparation phase (like translations), then you can simply assign a task branch. Instead of merging to stable, you merge to the task branch, and once the task is done, you merge the task branch to stable. An Example: Assume that you need to update translations before you release anything. (next part: init: you only need this once) When you want to do the first release which needs to be translated, you update to the revision from which you want to make the release and create the “translation” branch: hg update default; hg branch translation; hg commit -m "prepared the translation branch". All translators now update to the translation branch and do the translations. Then you merge it into stable: hg update stable; hg merge translation; hg ci -m "merged translated source for release". After the release you merge stable back into default as usual. (regular releases) If you want to start translating the next time, you just merge the revision to release into the translation branch: hg update translation; hg merge default; hg commit -m "prepared translation branch". Afterwards you merge “translation” into stable and proceed as usual. 

  10. If you want to adapt the model to multiple very distinct releases, simply add multiple release-branches (i.e. release-x). Then hg graft the changes you want to use from default or stable into the releases and merge the releases into stable to ensure that the relationship of their changes to current changes is clear, recorded and will be applied automatically by Mercurial in future merges11. If you use multiple tagged releases, you need to merge the releases into each other in order - starting from the oldest and finishing by merging the most recent one into stable - to record the same information as with release branches. Additionally it is considered impolite to other developers to keep multiple heads in one branch, because with multiple heads other developers do not know the canonical tip of the branch which they should use to make their changes - or in case of stable, which head they should merge to for preparing the next release. That’s why you are likely better off creating a branch per release, if you want to maintain many very different releases for a long time. If you only use tags on stable for releases, you need one merge per maintained release to create a bugfix version of one old release. By adding release branches, you reduce that overhead to one single merge to stable per affected release by stating clearly, that changes to old versions should never affect new versions, except if those changes are explicitely merged into the new versions. If the bugfix affects all releases, release branches require two times as many actions as tagged releases, though: You need to graft the bugfix into every release and merge the release into stable.12 

  11. If for example you want to ignore that change to an old release for new releases, you simply merge the old release into stable and use hg revert --all -r stable before committing the merge. 

  12. A rule of thumb for deciding between tagged releases and release branches is: If you only have a few releases you maintain at the same time, use tagged releases. If you expect that most bugfixes will apply to all releases, starting with some old release, just use tagged releases. If bugfixes will only apply to one release and the current development, use tagged releases and merge hotfixes only to stable. If most bugfixes will only apply to one release and not to the current development, use release branches. 

Creating nice logs with revsets in Mercurial

In the mercurial list Stanimir Stamenkov asked how to get rid of intermediate merges in the log to simplify reading the history (and to not care about missing some of the details).

Update: Since Mercurial 2.4 you can simply use
hg log -Gr "branchpoint()"

I did some tests for that and I think the nicest representation I found is this:

hg log -Gr "(all() - merge()) or head()"

This article shows examples for this.

What can Freenet do well already?

this just happened to me in the #freenet IRC channel at freenode.net (somewhat edited):

  • toad_1: what can freenet do well already? [18:38]

  • sharing and retrieving files asynchronously, freemail, IRC2, publishing sites without need of a central server, sharing code repositories [18:39]


  1. toad alias Matthew Toseland is the main developer of freenet. He tends to see more of the remaining challenges and fewer of the achievements than me - which is a pretty good trait for someone who builds a system to which we might have to entrust our basic right of free speech if the worls goes on like this. From a PR perspective it is a pretty horrible trait, though, because he tends to forget to tell people what freenet can already do well :) 

  2. To setup the social networking features of Freenet, have a look at the social networking guide 

Recipes for presentations with beamer latex using emacs org-mode

I wrote some recipes for creating the kinds of slides I need with emacs org-mode export to beamer latex.

Update: Read ox-beamer to see how to adapt this to work with the new export engine in org-mode 0.8.

PDF recipes The recipes as PDF (21 slides, 247 KiB)

org-mode file The org-mode sources (12.2 KiB)

Below is an html export of the org-mode file. Naturally it does not look as impressive as the real slides, but it captures all the sources, so I think it has some value.

Note: To be able to use the simple block-creation commands, you need to add #+startup: beamer to the header of your file or explicitely activate org-beamer with M-x org-beamer-mode.

«I love your presentation»:

PS: I hereby allow use of these slides under any of the licenses used by worg and/or the emacs wiki.

Open Letter to Julia Hilden on her article about pay-per-use

I just read your article on per use payments.

I think there are two serious flaws in per use payments:

(a) Good works of art need to last

As you stated correctly, I define myself partly through the media I "consume".

This does mean, that I want to have the assurance, that I can watch a great movie again a few years in the future.

Imagine this scenario:

  • I found a really great book, read it and got entranced.
  • It's 20 years later, now. and I want to read the book to my children.

Background of Freenet Routing and the probes project (GSoC 2012)

The probes project is a google summer of code project of Steve Dougherty intended to optimize the network structure of freenet. Here I will give the background of his project very briefly:

The Small World Structure

Minimal example for literate programming with noweb in emacs org-mode

If you want to use the literate programming features in emacs org-mode, you can try this minimal example to get started: Activate org-babel-tangle, then put this into the file noweb-test.org:

Minimal example for noweb in org-mode

* Assign 

First we assign abc:

#+begin_src python :noweb-ref assign_abc
abc = "abc"
#+end_src

* Use

Then we use it in a function:

#+begin_src python :noweb tangle :tangle noweb-test.py
def x():
  <<assign_abc>>
  return abc

print(x())
#+end_src

noweb-test.org

The ease of losing the spirit of your project by giving in to short-term convenience

Yesterday I said to my father

» Why does your whole cooperative have to meet for some minor legalese update which does not have an actual effect? Could you not just put into your statutes, that the elected leaders can take decisions which don’t affect the spirit of the statutes? «

He answered me

» That’s how dictatorships are started.«

With an Ermächtigungsbescheid.

I gulped a few times while I realized how easy it is to fall into the pitfalls of convenience - and lose the project in the process.

Install and setup infocalypse on GNU/Linux (script)

Install and setup infocalypse on GNU/Linux:

setup_infocalypse_on_linux.sh

Just download and run1 it via

wget http://draketo.de/files/setup_infocalypse_on_linux.sh_1_0.txt
bash setup_infocalypse*

This script needs a running freenet node to work!

In-Freenet-link: CHK@RZjy7Whe3vT3aEdox3pEG4fRbmRGsyuybPPhdvr7MoQ,g8YZO1~FAJM5suS7Uch06ugblVPE4YJd1rl15DxAwkY,AAMC--8/setup_infocalypse_on_linux.sh

The script allows you to get and setup the infocalypse extension with a few keystrokes to be able to instantly use the Mercurial DVCS for decentral, anonymous code-sharing over freenet.


  1. On systems based on Debian or Gentoo - including Ubuntu and many others - this script will install all needed software except for freenet itself. You will have to give your sudo password in the process. Since the script is just a text file with a set of commands, you can simply read it to make sure that it won’t do anything evil with those sudo rights

Custom link completion for org-mode in 25 lines (emacs)

Update (2013-01-23): The new org-mode removed (org-make-link), so I replaced it with (concat) and uploaded a new example-file: org-custom-link-completion.el.
Happy Hacking!

1 Intro

I recently set up custom completion for two of my custom link types in Emacs org-mode. When I wrote on identi.ca about that, Greg Tucker-Kellog said that he’d like to see that. So I decided, I’d publish my code.

Reducing the Python startup time

The python startup time always nagged me (17-30ms) and I just searched again for a way to reduce it, when I found this:

The Python-Launcher caches GTK imports and forks new processes to reduce the startup time of python GUI programs.

Python-launcher does not solve my problem directly, but it points into an interesting direction: If you create a small daemon which you can contact via the shell to fork a new instance, you might be able to get rid of your startup time.

The generation of cultural freedom

I am part of a generation who experienced true cultural freedom - and who experienced that freedom being destroyed.

We had access to the largest public library which ever existed and saw it burned down for lust for control.1


  1. I saw the Napster burn, I saw Gnutella burn, I saw edonkey burn, I saw Torrentsites burn, I saw one-click-hosters burn and now I see Youtube burn with blocked and deleted videos - even those from the artists themselves. 

El Kanban Org: parse org-mode todo-states to use org-tables as Kanban tables

Kanban for emacs org-mode.

Update (2013-04-13): Kanban.el now lives in its own repository: on bitbucket and on a statically served http-repo (to be independent from unfree software).

Update (2013-04-10): Thanks to Han Duply, kanban links now work for entries from other files. And I uploaded kanban.el on marmalade.

Some time ago I learned about kanban, and the obvious next step was: “I want to have a kanban board from org-mode”. I searched for it, but did not find any. Not wanting to give up on the idea, I implemented my own :)

The result are two functions: kanban-todo and kanban-zero.

“Screenshot” :)

TODODOINGDONE
Refactor in such a way that the
let Presentation manage dumb sprites
return all actions on every command:
Make the UiState adhere the list of
Turn the model into a pure state

Read your python module documentation from emacs

I just found the excellent pydoc-info mode for emacs from Jon Waltman. It allows me to hit C-h S in a python file and enter a module name to see the documentation right away. If the point is on a symbol (=module or class or function), I can just hit enter to see its docs.

pydoc in action

Exploring the probability of successfully retrieving a file in freenet, given different redundancies and chunk lifetimes

In this text I want to explore the behaviour of the degrading yet redundant anonymous file storage in Freenet. It only applies to files which were not subsequently retrieved.

Every time you retrieve a file, it gets healed which effectively resets its timer as far as these calculations here are concerned. Due to this, popular files can and do live for years in freenet.

The “Apple helps free software” myth

→ Comment to “apple supports a number of opensource projects. Webkit and CUPS come to mind”.

Apple supports a number of copyleft projects, because they have to.

Write programs you can still hack when you feel dumb

I just read the post Hyperfocus and balance of Arc Riley from PySoy who talks about trying to get to the Hyperfocus state without endangering his health.

Effortless password protected sharing of files via Freenet

Often you want to exchange some content only with people who know a given password and make it accessible to everyone in your little group but invisible to the outside world.

Until yesterday I thought that problem slightly complex, because everyone in your group needs a given encryption program, and you need a way to share the file without exposing the fact that you are sharing it.

Then I learned two handy facts about Freenet:

  • <ArneBab> evanbd: If I insert a tiny file without telling anyone the key, can they get the content in some way?
    <evanb
Inhalt abgleichen