– Juan Gabriel Covas - 2011

Me propuse aprender a gestionar un servidor Subversion, a montarlo sobre protocolo svn+ssh para hacerlo lo más seguro posible, y soportar que hubiera ciertos “repositorios” públicos y otros privados. La única buena documentación que encontré sobre la complicación de montarlo bien debido a los permisos bajo Linux era para Ubuntu en su momento, así que es el único entorno donde lo he podido probar bien en casos reales (varias personas usándolo sin problemas).

Si hoy tuviera que elegir, bastaría usar VisualSVN bajo Windows que soporta SSL bajo un Apache en el puerto especial que elijas, aunque sin soporte para los juguetes que proporcionan svnnotify y probablemente otros.

Estas son mis notas, mezcladas en inglés y español, junto con otras que no son mías. Las he utilizado ya 3 veces para montar un servidor Subversion svn+ssh sobre Ubuntu, aunque aviso que no es una guía “paso a paso”.

$ sudo apt-get install subversion

(For ubuntu 12 and others will install subversion 1.6.x. If you want subversion 1.7.x or higher use wandisco setup bash script)

Let's assume we want “public” and “devel” (private) repos:

$ sudo mkdir -p /usr/local/svn/public/repos
$ sudo mkdir -p /usr/local/svn/devel/repos

Now, you need to set some access permissions on those directories. You only want to allow certain users of your system (that is, yourself and the other developers) to access the repositories, so add a new group for those users. Name the group svn.

$ sudo groupadd svn

Then, change the group ownership of /usr/local/svn/[public|devel]/repos to the new group using the chgrp command:

$ sudo chgrp svn /usr/local/svn/public/repos /usr/local/svn/devel/repos

The members of the svn group also need write access to the repos directory, so use chmod to add the write permission for the group:

$ sudo chmod g+w /usr/local/svn/public/repos /usr/local/svn/devel/repos

Additionally, you need to make sure that all new files and directories created in the repos directory (in other words, anything committed to the repositories) will also be owned by the group. To accomplish this, use chmod again to set the set-group-ID bit (or SETGID bit) on the directory, which causes any file created inside it to have the same group ownership as the directory itself. Effectively, everything in repos will belong to the svn group.

$ sudo chmod g+s /usr/local/svn/public/repos /usr/local/svn/devel/repos

OK, so you now have the repositories directory with proper permissions, ready to be used by the svn group.

Go ahead and add your own user in the server to the svn group:

a) If your user already exist:

$ sudo usermod -a -G svn juanga

b) Creating a new user:

$ sudo adduser <newlogin>
$ sudo usermod -a -G svn <newlogin>

However, your new group membership will not be effective for the current session, so you need to log out and log back in. When you're back, you can verify that your account is recognized as a member of the svn group:

$ groups
juanga adm dialout cdrom plugdev lpadmin admin sambashare svn

If other developers already have user accounts on your server, add them to the group too:

$ sudo usermod -a -G svn toni

(To create a new user including it directly to svn group, use this: $ sudo useradd -G svn <username>)

It's good to create a svnadmin user using the script 'adduser':

$ sudo adduser svnadm

Add to group “svn”:

$ sudo usermod -a -G svn svnadm

If we want this user to be able to sudo (In ubuntu there is no wheel group):

$ sudo usermod -a -G admin svnadm

(THIS IS NOT A GOOD IDEA, if we later want to run svnserve as svnadm user instead of root user!)

Login with $ su - svnadm

Add to his .bashrc:

umask 002

PATH_SVN_PUBLIC="/usr/local/svn/public/repos"
PATH_SVN_DEVEL="/usr/local/svn/devel/repos"

“umask” command sets the new file mode creation mask which controls the default permissions of any new file that you create. The default value is 022 and it corresponds to read/write permissions for the file owner, and read permissions for the group and others. The new value, 002, also gives write permissions to the group, which is just what you need.

(JUST LEARNING, DOT NOT USE ON PRODUCTION)

You can now create a repository. In the following steps, I'll demonstrate how to create a simple test repository containing one text file, and how to check out and commit files. If you're not familiar with Subversion, then this could be a good exercise to learn the basics. Otherwise, you can skip all the test checkouts and commits and just create the repository for your project.

The repository will be a subdirectory in the repos directory, and will have its group ownership set to svn (thanks to the chmod g+s you did earlier).

However, that's not all – you also need to make sure the repository will be group writable, so that the other members of the svn group will be able to commit files. To do this, set the umask to 002:

$ umask 002

This command sets the new file mode creation mask which controls the default permissions of any new file that you create. The default value is 022 and it corresponds to read/write permissions for the file owner, and read permissions for the group and others. The new value, 002, also gives write permissions to the group, which is just what you need.

Create the repository using the svnadmin command:

$ svnadmin create /usr/local/svn/public/repos/test

And set back the default umask:

$ umask 022

So you now have an empty repository, waiting for you to commit something to it. But, before you do this, you need to check out the current version (i.e., the empty directory) to create a working copy.

Let's say you want it under /home/youruser

$ cd ~
$ svn checkout file:///usr/local/svn/repos/test mytestworkingcopy
Checked out revision 0.

The working copy has been checked out to a new directory named mytestworkingcopy. Go ahead and create a simple “hello world” text file in that directory:

$ cd mytestworkingcopy

$ echo 'Hello, World!' > hello.txt

Then, add it to version control with the svn add command:

$ svn add hello.txt
A         hello.txt

Finally, commit it using svn commit:

$ svn commit -m "Added a 'hello world' text file."
Adding         hello.txt
Transmitting file data .
Committed revision 1.

The hello.txt file is now in the repository.

(TESTING A BIT TO LEARN)

(UNENCRYPTED, REQUIRE SVN USER/PASSWORD, USEFUL FOR PUBLIC REPOS)

Remote repository access with the svn protocol requires you to use svnserve, a Subversion server program. Each repository has a svnserve configuration file (stored in the conf subdirectory) which controls how the repository can be accessed with svnserve.

First, create a passwords file that lists the users of the repository and their passwords. This will be a common passwords file for your development team and you will be able to use it with multiple repositories.

$ sudo nano /usr/local/svn/public/passwd-team

Here's a sample passwords file. Each line (except the first one, which is the configuration section name) defines a user name and the corresponding password.

[users]
testuser = somepassword
user123 = anotherpassword

Since the passwords are stored unencrypted, it's important that you protect the passwords file by setting the proper permissions. The file should not be readable by anyone except the owner (which is root), so change its mode to 600:

$ sudo chmod 600 /usr/local/svn/public/passwd-team

Then, open the svnserve configuration file in the test repository:

$ nano /usr/local/svn/public/repos/test/conf/svnserve.conf

There's probably some default configuration in the file, but you can just remove everything and enter this:

[general]
anon-access = none
password-db = /usr/local/svn/public/passwd-team
realm = public-repositories

The anon-access = none line denies access to the repository to unauthenticated users (by default, they are allowed read access, so they can do checkouts).

The password-db setting tells svnserve where to look for the passwords file when authenticating users, and the realm setting defines the name of the authentication realm.

OK, the configuration is ready, so you can now launch svnserve.

Do it from a new console:

$ sudo svnserve -d --foreground -r /usr/local/svn/public/repos

The command-line options tell svnserve to run in daemon mode (-d) as a foreground process (–foreground), and to look for repositories in the repos dir that was created earlier (-r /usr/local/svn/repos). Normally the program should be running in the background (that's what daemon processes do), but at this moment you only need to test it, so it's more convenient to run it in the foreground, where you can easily kill it with Ctrl+C.

Now, try accessing the repository using the svn protocol. You can try it on another machine over the network, or on the same computer (in another terminal). In the latter case, make sure you're not doing the checkout in the same directory where the previous test working copy was checked out, because it won't work – either delete the test directory, or cd to some other location.

Enter the following svn checkout command, replacing 192.168.1.2 with the IP address of your Subversion server (if you're testing on the same machine, you can use 127.0.0.1):

Note that since svnserve was told to run under /usr/local/svn/public/repos, only *public repos* will be accessible through svn:// protocol

  ----------------------------------+
                                    |
                                    V
  $ svn checkout svn://your-svn-server-IP/test --username testuser

The server will ask you for password:

Authentication realm: <svn://your-svn-client-IP:3690> public-repositories
Password for 'testuser': 

Then, it proceeds with the checkout:

A    test/hello.txt
Checked out revision 1.

And there's your working copy. Now, check if it works the other way – try modifying the file and committing it back to the repository. Open hello.txt with a text editor and add some text:

 $ cd test
$ gedit hello.txt

When you're done, commit it:

$ svn commit -m “Modified the hello.txt file.”

Sending        hello.txt
Transmitting file data .
Committed revision 2.

Sweet, it works both ways.

(Ubuntu init script)

If you plan on using svnserve in the long run, you probably don't want to start it from the command-line every time the server is rebooted. The proper way to start system services is with init scripts located in the /etc/init.d directory.

The Subversion package for Ubuntu does not include an init script, so you can use the following one:

Save it as /etc/init.d/svnserve

NOTE that this script contains the DAEMON_ARGS where we setup the directory /usr/local/svn/public/repos, so only *public repos* will be accessible through svn:// protocol

* CHANGING DEFAULT PORT *

If you want svnserve to listen on another port than default 3690, add –listen-port=XXXX to DAEMON_ARGS below:

* ADDING LOGS *

If you want svnserve to log: add –log-file=/var/log/svnserve.log to DAEMON_ARGS below

#! /bin/sh
### BEGIN INIT INFO
# Provides:          svnserve
# Required-Start:    $local_fs $syslog $remote_fs
# Required-Stop:     $local_fs $syslog $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start svnserve
### END INIT INFO

# Author: Michal Wojciechowski <odyniec@odyniec.net>

PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="svnserve"
NAME=svnserve
DAEMON=/usr/bin/$NAME
DAEMON_ARGS="-d -r /usr/local/svn/public/repos"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME

[ -x "$DAEMON" ] || exit 0

[ -r /etc/default/$NAME ] && . /etc/default/$NAME

. /lib/init/vars.sh

. /lib/lsb/init-functions

do_start()
{
	start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
		|| return 1
	start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
		$DAEMON_ARGS \
		|| return 2
}

do_stop()
{
	start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
	RETVAL="$?"
	[ "$RETVAL" = 2 ] && return 2
	start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON
	[ "$?" = 2 ] && return 2
	rm -f $PIDFILE
	return "$RETVAL"
}

case "$1" in
  start)
	[ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
	do_start
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  stop)
	[ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
	do_stop
	case "$?" in
		0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
		2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
	esac
	;;
  restart|force-reload)
	log_daemon_msg "Restarting $DESC" "$NAME"
	do_stop
	case "$?" in
	  0|1)
		do_start
		case "$?" in
			0) log_end_msg 0 ;;
			1) log_end_msg 1 ;; # Old process is still running
			*) log_end_msg 1 ;; # Failed to start
		esac
		;;
	  *)
	  	# Failed to stop
		log_end_msg 1
		;;
	esac
	;;
  *)
	echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
	exit 3
	;;
esac

exit 0

Make the new init script executable:

$ sudo chmod +x /etc/init.d/svnserve

If you chose anything other than /usr/local/svn/public/repos for the repositories directory you want to make available through svnserve, make sure to change the path in the init script.

Next, run update-rc.d to install the script:

$ sudo update-rc.d svnserve defaults
Adding system startup for /etc/init.d/svnserve ...
/etc/rc0.d/K20svnserve -> ../init.d/svnserve
[...]

And that's it – svnserve will be started automatically when your system boots up.

Instead to wait for the next reboot, start it now manually using:

$ sudo /etc/init.d/svnserve start

You can check it's running with:

$ ps -ef | grep svnserve

As usual, you can do:

$ sudo /etc/init.d/svnserve start
$ sudo /etc/init.d/svnserve stop
$ sudo /etc/init.d/svnserve restart

Or use “sudo service svnserve” on latest Ubuntu versions.

(NOT tested)

Runs svnserve as 'svn' user

# vi /etc/xinetd.d/svn

service svn
{
port = 3690
socket_type = stream
protocol = tcp
wait = no
user = svnadm
server = /usr/bin/svnserve.bin
server_args = -i -r /home/svn
}

where svnserve runs as user svn, and serves repositories off /home/svn.

We restart xinetd for the changes to take effect.

# /etc/init.d/xinetd restart

(USEFUL FOR PRIVATE/DEVEL REPOS)

Setting up your Subversion server for svn+ssh access is simple, as it doesn't even require using the svnserve program. Assuming you have a SSH server running on the Subversion server machine, and the other developers can login to it, you don't have to configure (almost) anything – just set up the repository.

You can just go ahead and check out the test project we did for the public repos.

The checkout operation is slightly different with the svn+ssh access method. First, you must specify the full path to the repository in the checkout URL:

$ svn checkout svn+ssh://svn-server-IP/usr/local/svn/public/repos/test --username testuser

Then, when the server asks you for a password, you need to enter the user's SSH password, not the one from the passwd-team file.

testuser@192.168.1.2's password:

And there it goes:

A test/hello.txt
Checked out revision 2.

From here, you can use your working copy the same way as with the svn protocol.

TIP for the server: Accelerating SSH connection from server-side: There may be a long delay if the SSH server is trying to resolve the client's hostname and DNS are having trouble. Avoid DNS resolution of clients just by adding: “UseDNS no” to your /etc/ssh/sshd_config file in your svn server box (and restart ssh daemon with: sudo /etc/init.d/ssh restart) in Ubuntu.

RENAMING SVNSERVE DAEMON:

We want, on the *same* svn server box:

• One “public”, svn:// protocol access to repositories, say to directory “…/svn/public/repos” that holds several repos. So 'svnserve' has to be configured to serve only from THAT 'public repos' directory.
  • This way you can share an 'easy to access' svn server, having projects that does not need encription, extra security, etc.
  • This also allows us to set 'anon-access: read' in svnserve.conf for some repositories, some not and so on, and also extra, standard authenticated users for write.

• One “private”, svn+ssh:// protocol access to repositories, say to directory “…/svn/devel/repos” that holds several sensible, private repos.
  • Goals:
    • Having a secure set of private repositories being SVN+SSH the only allowed protocol to access them.
    • Using SSH only, encryption is enforced for authentication AND all data transmitted for those repositories.
    • Each developer will NEED an unprivileged, user account in the svn server.
    • Enforcing the use of dsa keys + pass-phrases, and not passwords, adds extra security.

So:

Accessing through the plain svn:// protocol will try to connect to the 'svnserve' daemon, that should be started and listening on host:3690 (by default). We'll configure 'svnserve' daemon to “public repos” directories only, and we can further restrict access by using svnserve.conf file on each repo Accessing through the svn+ssh:// protocol does *NOT* try to connect to the running 'svnserve' daemon process: instead, the SSH starts a new 'svnserve' running it as the SSH authenticated user, and then access the “devel repos” only

Exactly, from the TortoiseSVN docs for svnserve, we find that using SVN+SSH: […] svnserve is not run as a daemon process, rather, the secure shell starts svnserve for you, running it as the SSH authenticated user. To enable this, you need a secure shell daemon on your server“.

PROBLEM: The svn+ssh protocol will expose *full* paths when you give the URL to the repository (eg. /usr/local/svn/devel/repos in your svn+ssh://user@host:port/full/path/)

To workaround this, do the following:

$ sudo mv /usr/bin/svnserve /usr/bin/svnserve.bin

Now we create a wrapper script

$ sudo nano /usr/bin/svnserve

#!/bin/sh
exec /usr/bin/svnserve.bin --log-file /var/log/svn+ssh/log.txt -r /usr/local/svn/devel/repos "$@"

Set permissions on the new svnserve wrapper bash script:

sudo chmod u+wrx,g+rx-w,o+xr-w /usr/bin/svnserve

And this way a valid SVN+SSH URL will be: svn+ssh://user@host/REPONAME ←– instead of /usr/local/svn/devel/repos/REPONAME

• Remember to change the script: /etc/init.d/svnserve to execute “svnserver.bin”
• Change the DAEMON shell var inside the init.d script and will work.
• If already running svnserve: ps -ef | grep svnserve AND kill pid
• Start the new .bin daemon to see if it works: sudo /etc/init.d/svnserve start

Remember that if you use –log-file in daemon to write a log file, for example:

exec /usr/bin/svnserve.bin --log-file /var/log/svn+ssh/log.txt -r /usr/local/svn/devel/repos "$@"

…you need to prepare the logs subdirectory this way:

Create the new logs subdirectory for subversion:

$ sudo mkdir -p /var/log/svn+ssh

Then change the group ownership for our subversion logs subdirectory:

$ sudo chgrp svn /var/log/svn+ssh

Members of the svn group also need write access:

$ sudo chmod g+w /var/log/svn+ssh

Make sure that all new files and directories created in the new subversion logs directory will also be owned by the svn group:

$ sudo chmod g+s /var/log/svn+ssh

CHECK that everything is OK for other users:

This perms are INCORRECT:

-rw-r--r--  1 juanga svn  4793 Jan 14 20:17 log.txt

Fix with:

$ sudo chmod g+w /var/log/svn+ssh/log.txt
-rw-rw-r--  1 juanga svn  4793 Jan 14 20:17 log.txt

(Just testing)

Assuming that you have:

1. a private/public key pair to use it in windows to be able to connect to a remote unix machine using Putty
2. your public key is added to the authorized_keys file in the subversion, linux SERVER

You need to configure Subversion to use “plink” (the command-line putty SSH client for Windows) for connecting using “svn+ssh” URI:

• Install the full “putty” windows installer package from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
• Not really needed: Add the putty directory to windows PATH env. variable, so “plink” is accessible from any place
• We need to install a windows command-line subversion client, so we can install SlikSVN (or any other SVN commandline binaries) from http://www.sliksvn.com/en/download

Edit the subversion “config” file:

You can found it at:

C:\Users\Juanga\AppData\Roaming\Subversion\config

or

C:\Documents and Settings\user\Application Data\Subversion\config

Locate the section named [tunnels]

Add the following line:

ssh=c:/path/to/plink.exe -i c:/path/to/your/valid/key/for/putty/key.ppk 

In my case:

ssh = C:/path/to/plink.exe -i C:/path_to/private_id_dsa.ppk

Optionally you can add -C to plink.exe to enable compression (less data transmitted over the network)

You are done, it should work now. Let’s give a try:

$ svn co svn+ssh://user@server/path/to/svn/repository dirname

If not, try adding -v option to plink.exe to see verbose debug of SSH connection…

On Unix systems, it is usually a great time-saver to use shell variables to store repo URLS, rather than retyping them everywhere.

Eg. putting this in your ~/.bashrc file:

PATH_SVN_PUBLIC="/usr/local/svn/public/repos"
PATH_SVN_DEVEL="/usr/local/svn/devel/repos"

To accessing these dirs using cd $PATH_SVN_PUBLIC

To create a 'real' repo under our special crafted dirs when preparing the server:

$ umask 002
$ svnadmin create $PATH_SVN_PUBLIC/mynewrepo
$ umask 022

$ svn mkdir file://$PATH_SVN_PUBLIC/mynewrepo/trunk \
            file://$PATH_SVN_PUBLIC/mynewrepo/branches \
            file://$PATH_SVN_PUBLIC/mynewrepo/tags \
      -m "creating initial repository layout"

“Repository Permissions” Section

Reference: “Practical Subversion” Book, section “Backups”

Others:

OPTION #1

Hay varias formas de hacer backups de svn, pero esta es la única que he podido utilizar que me permita hacerlo en caliente sobre un subversion funcionando y que me permite llevar todo el historico a otro subversion en una maquina remota

$ svnadmin dump --deltas /repo | bzip2 | tee dump.bz2 | md5sum >dump.md5

Ahora podemos copiar los ficheros dump.bz2 y dump.md5 a la maquina remota donde hemos creado un nuevo repositorio de subversion

Para restaurar el repositorio en una maquina nueva

md5sum -c dump.md5 <dump.bz2
svnadmin create newrepo
bzcat dump.bz2 | svnadmin load newrepo

Pero si lo que queremos es hacer un restore en local de una copia, digamos de ayer, con el mismo nombre del repositorio existente, hay que hacer un paso extra

# mv /path/to/repo /path/to/repo-original
# mkdir /path/to/repo
# svnadmin create /path/to/repo
# bzcat dump.bz2 | svnadmin load /path/to/repo

OPTION #2

El backup y restore de los repositorios de Subversion, pueden llevarse a cabo de manera fácil y rápida a través de las herramientas de administración por consola:

# svnadmin dump /path/to/repo > svn-backup

Restore del repositorio:

Si hemos de restaurar en un entorno nuevo es fácil, creamos el repositorio y restauramos

# svnadmin create newrepo

# svnadmin load /path/to/newrepo < svn-backup

Pero si lo que queremos es hacer un restore en local, con el mismo nombre del repositorio existente, hay que hacer un paso extra

# mv /path/to/repo /path/to/repo-original

# mkdir /path/to/repo
# svnadmin create /path/to/repo
# svnadmin load /path/to/repo < svn-backup