Welcome to http://www.marssoft.de/
 
Wednesday, 13th November 2019 16:31:02 (GMT+1) 

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

guides:cvs_to_subversion [2014/04/02 22:39] (current)
Line 1: Line 1:
 +
 +==== Subversion short Introduction ====
 +
 +Since 2009.07.04, our source code repository is stored in subversion on kontiki2.bitplane.com.
 +To check out the current trunk into the new directory ''​mysources'',​ do
 +<​code>​
 +svn checkout svn://​kontiki2.bitplane.com/​_REPOSITORY_SVN/​trunk mysources
 +</​code>​
 +
 +If you only need a subdirectory of the _REPOSITORY_SVN,​ choose the corresponding call from:
 +<​code>​
 +svn ls svn://​kontiki2.bitplane.com/​_REPOSITORY_SVN/​trunk
 +svn checkout svn://​kontiki2.bitplane.com/​_REPOSITORY_SVN/​trunk/<​directory>​ <​directory>​
 +</​code>​
 +
 +Currently, we have the following Repositories available:
 +<​code>​
 +bitbase
 +bitplane
 +bitplane2003
 +bitplane2005
 +bitplane2008
 +imarisXT
 +installation
 +manual
 +matlab
 +neuronTracer
 +software
 +wda
 +</​code>​
 +
 +
 +==== Conversion from CVS to Subversion - 2009.07.04 ====
 +
 +   * A good cvs2svn introduction can be found here: http://​cvs2svn.tigris.org/​cvs2svn.html
 +   * details specific to OpenSUSE can be found here: http://​en.opensuse.org/​Subversion
 +
 +
 +To convert the CVS repository to Subversion, first install subversion and cvs2svn on kontiki2
 +via yast: install subversion cvs2svn
 +
 +=== Run the Subversion service //​svnserve//​ ===
 +
 +The service //​svnserve//​ should not run as root user. The startup script /​etc/​init.d/​svnserve expects a user/group named '​svn',​ but this user/group must be created before first use. First, we check it does not already exist (this guide may be outdated by the time you read it!)
 +<​code>​
 +grep svn /etc/passwd /etc/group
 +</​code>​
 +
 +If the grep does not return a hit, you should create a group and user for //​svnserve//,​ named //svn//. It is a good idea to disable login for this user by setting the shell to /bin/false.
 +<​code>​
 +sudo groupadd --system svn
 +sudo useradd --system --gid svn svn
 +sudo usermod --shell /bin/false svn
 +</​code>​
 +
 +Then we create a new, empty subversion repository. First, the directory is created. Then use svnadmin to create the configuration skeleton. We do not use the default OpenSUSE location /srv/svn/, but rather the Bitplane location /​development/​ for the new repository _REPOSITORY_SVN. Make sure that the files of the repository are accessible by the user "​svn"​. It would be best to give exclusive read and write access of the whole repository directory tree only to the user svn.
 +<​code>​
 +sudo mkdir -p /​development/​_REPOSITORY_SVN
 +sudo svnadmin create /​development/​_REPOSITORY_SVN
 +sudo chown -R svn:svn /​development/​_REPOSITORY_SVN
 +sudo ls -la /​development/​_REPOSITORY_SVN
 +
 +drwxr-xr-x ​ 2 svn  svn         128 2009-07-04 11:38 conf
 +drwxr-xr-x ​ 2 svn  svn          48 2009-07-04 11:38 dav
 +drwxr-sr-x ​ 5 svn  svn         256 2009-07-04 11:38 db
 +-r--r--r-- ​ 1 svn  svn           2 2009-07-04 11:38 format
 +drwxr-xr-x ​ 2 svn  svn         360 2009-07-04 11:38 hooks
 +drwxr-xr-x ​ 2 svn  svn         104 2009-07-04 11:38 locks
 +-rw-r--r-- ​ 1 svn  svn         229 2009-07-04 11:38 README.txt
 +</​code>​
 +
 +To change the default repository location that svnserve should serve, and the access permissions,​ use YAST. Go to "​Network Services"​ -> "​Network Services (xinetd)"​ -> "​svn"​ -> "​Edit"​ -> "​Server Arguments"​. There you can set the repository root path, and the read-write-access,​ by changing the arguments from
 +<​code>​
 +# --read-only --root=/​srv/​svn/​repos --inetd
 +</​code>​
 +to
 +<​code>​
 +--root=/​development/​_REPOSITORY_SVN --inetd
 +</​code>​
 +
 +Then enable the service by using the ''​Toggle Status (On or Off)''​ option in YAST. To configure the user access rights for the repository, continue with the next section.
 +
 +
 +=== Subversion Configuration and (Password) Authentication ===
 +
 +For subversion, all configuration files (''​svnserve.conf'',​ ''​passwd'',​ ''​authz''​) are located in the folder [repository]/​conf/​. The default permissions are world-readable,​ it is a very good idea to change them to something as strictly as possible.
 +<​code>​
 +sudo chmod u=rw,g=,o= /​development/​_REPOSITORY_SVN/​conf/​*
 +sudo chmod u=rwx,g=,o= /​development/​_REPOSITORY_SVN/​conf
 +ls -la /​development/​_REPOSITORY_SVN/​conf/​
 +total 12
 +-rw------- 1 svn svn  684 2009-07-04 11:38 authz
 +-rw------- 1 svn svn  309 2009-07-04 11:38 passwd
 +-rw------- 1 svn svn 1457 2009-07-04 11:38 svnserve.conf
 +</​code>​
 +
 +The svnserve.conf controls access permissions and the location of the other two configuration files ''​passwd''​ and ''​authz''​. For the Bitplane setup, we want no anonymous access (''​anon-access = none''​),​ and we grant write permissions to authenticated users (''​auth-access = write''​). The password database file is located in the default location in the conf/ directory (''​password-db = passwd''​). For a very detailed control of access permissions,​ the authz-db file could be used, but that seems to be overkill for our simple setup (''# ​ authz-db''​). For a more intuitive feel, we can set the password realm to Bitplane (''​realm = Bitplane''​).
 +<​code>​
 +sudo cat /​development/​_REPOSITORY_SVN/​conf/​svnserve.conf
 +[general]
 +### These options control access to the repository for unauthenticated
 +### and authenticated users. ​ Valid values are "​write",​ "​read",​
 +### and "​none"​. ​ The sample settings below are the defaults.
 +anon-access = none
 +auth-access = write
 +###
 +### The password-db option controls the location of the password
 +### database file.
 +password-db = passwd
 +###
 +### The authz-db option controls the location of the authorization
 +### rules for path-based access control.
 +#  authz-db = authz
 +###
 +### This option specifies the authentication realm of the repository.
 +realm = Bitplane
 +</​code>​
 +
 +Password authentication is done in plaintext, using the file ''​passwd''​
 +<​code>​
 +sudo cat /​development/​_REPOSITORY_SVN/​conf/​passwd
 +[users]
 +emmenlau = my_plaintext_passwd
 +</​code>​
 +
 +It is now already possible to log into the Subversion server and list the available modules.
 +<​code>​
 +svn ls svn://​kontiki2.bitplane.com/​
 +Authentication realm: <​svn://​kontiki2.bitplane.com:​3690>​ Bitplane
 +Password for '​emmenlau':​
 +</​code>​
 +
 +This is partially an empty-set-joke :-)
 +
 +
 +=== Creating the CVS dump, Importing into Subversion ===
 +
 +In order for the CVS repository to stay available unmodified, one can remove write permissions by adding an empty ''​writers''​ file.
 +<​code>​
 +if test -e /​development/​_REPOSITORY/​CVSROOT/​writers ; then
 +  sudo mv /​development/​_REPOSITORY/​CVSROOT/​writers /​development/​_REPOSITORY/​CVSROOT/​writers.bak
 +fi
 +sudo touch /​development/​_REPOSITORY/​CVSROOT/​writers && \
 +sudo chmod 644 /​development/​_REPOSITORY/​CVSROOT/​writers
 +</​code>​
 +
 +When trying a commit now, CVS will report
 +<​code>​
 +commit -l -m "​whitespace changes only" thirdparty/​zlib-1.2.3.pro
 +cvs [server aborted]: "​commit"​ requires write access to the repository
 +</​code>​
 +
 +The goal now is to generate a CVS dump containing all previous CVS operations from the CVS repository. This dump could now be generated from the readonly CVS repository, but we might want to make changes to the repo before creating the dump. Therefore we create a copy first.
 +<​code>​
 +sudo mkdir -p /​development/​tmp/​ && \
 +sudo cp -par /​development/​_REPOSITORY /​development/​tmp/​
 +</​code>​
 +
 +The only change we want to make is to do some cleanup, then rename the ''​bitplane''​ directory to ''​bitplane1993'',​ and to rename the ''​bitplane2008''​ directory back to its basic form ''​bitplane''​.
 +<​code>​
 +sudo chown -R root:root /​development/​tmp/​_REPOSITORY
 +sudo find /​development/​tmp/​_REPOSITORY/​ -name .DS_Store -exec rm -f \{\} \;
 +sudo mv /​development/​tmp/​_REPOSITORY/​bitplane /​development/​tmp/​_REPOSITORY/​bitplane1993
 +sudo mv /​development/​tmp/​_REPOSITORY/​bitplane2008 /​development/​tmp/​_REPOSITORY/​bitplane
 +</​code>​
 +
 +Now would also be the time to remove unwanted directories from the tree, but we do not do that here.
 +<​code>​
 +# rm -fr /​development/​tmp/​_REPOSITORY/​installation
 +</​code>​
 +
 +There is a bug in the CVS repository that needs to be addressed before the dump can be created. The symbol ''​NeuronTracerInitial''​ is multiply defined in some files. Here is a call that removes (arbitrarily chosen) one of the symbols.
 +<​code>​
 +sudo grep --binary-files=text -rn "​NeuronTracerInitial.*NeuronTracerInitial"​ \
 +    /​development/​tmp/​_REPOSITORY/​neuronTracer \
 +    |cut -d':'​ -f1 \
 +    |while read FILENAME ; do \
 +         sudo perl -pi -e '​s/​NeuronTracerInitial.*(NeuronTracerInitial)/​$1/​g'​ "​$FILENAME"​ ; \
 +     done
 +sudo grep --binary-files=text -rn "​NeuronTracerInitial:​1.1.1.1$"​ \
 +    /​development/​tmp/​_REPOSITORY/​neuronTracer \
 +    |cut -d':'​ -f1 \
 +    |while read FILENAME ; do \
 +         sudo perl -pi -e '​s/​\s+NeuronTracerInitial:​1.1.1.1\s+//​g'​ "​$FILENAME"​ ; \
 +     done
 +</​code>​
 +
 +The CVS repository can now be exported into a set of operations that created it. To generate the Subversion dump file, containing all commits and changes, run cvs2svn on the repository structure. There is a potential line ending problem involved in this process. If binary files have not been added with the CVS ''​-kb''​ flag, the line endings in the binary files might have been converted. This is an issue when porting the file to Subversion, as the latter does not usually "​destroy"​ binary files in this way. One can use the cvs2svn flags ''​--eol-from-mime-type''​ to convert line endings automatically. This should not be required at Bitplane. First, a test-run (''​--dry-run''​):​
 +<​code>​
 +sudo rm -f /​development/​tmp/​_REPOSITORY.dump
 +sudo cvs2svn --encoding="​ASCII"​ --encoding="​UTF-8"​ --encoding="​ISO-8859-1"​ \
 +    --dumpfile=/​development/​tmp/​_REPOSITORY.dump \
 +    /​development/​tmp/​_REPOSITORY 2>&​1|tee /​tmp/​cvs2svn.log
 +
 +cvs2svn Statistics:
 +------------------
 +Total CVS Files: ​            59080
 +Total CVS Revisions: ​       165571
 +Total Unique Tags:              24
 +Total Unique Branches: ​         18
 +CVS Repos Size in KB:     ​11994524
 +Total SVN Commits: ​          16325
 +First Revision Date:    Mon Aug 18 16:11:20 1997
 +Last Revision Date:     Fri Jul  3 17:20:09 2009
 +------------------
 +Timings:
 +------------------
 +pass 1:   518 seconds
 +pass 2:     0 seconds
 +pass 3:    23 seconds
 +pass 4:     0 seconds
 +pass 5:    15 seconds
 +pass 6:    46 seconds
 +pass 7:     1 second
 +pass 8:     1 second
 +pass 9:  2573 seconds
 +total: ​  3179 seconds
 +</​code>​
 +
 +Once the dump file has been successfully generated, it can be imported into the Subversion repository.
 +<​code>​
 +cd /​development/​
 +sudo tar -czf _REPOSITORY_SVN_2009.07.04.tar.gz _REPOSITORY_SVN
 +sudo svnadmin load /​development/​_REPOSITORY_SVN < /​development/​tmp/​_REPOSITORY.dump
 +</​code>​
 +
 +
 +----
 +FIXME the svndumpfilter produced a dump that could not be imported into subversion. Do not use it.
 +
 +The dump can be optimized by piping it through svndumpfilter to remove empty revisions, remove the CVSROOT directory, and to fix the ordering of revisions if needed.
 +<​code>​
 +sudo chmod 777 /​development/​tmp
 +sudo cat /​development/​tmp/​_REPOSITORY.dump|svndumpfilter \
 +    --drop-empty-revs \
 +    --renumber-revs \
 +    exclude trunk/​CVSROOT \
 +    >/​development/​tmp/​_REPOSITORY.dump.filter
 +</​code>​
 +----
 +FIXME the rest of this document is unneeded, and only here for reference.
 +----
 +
 +=== Merge several different subversion repositories into a single one ===
 +
 +See http://​svnbook.red-bean.com/​en/​1.4/​svn.reposadmin.maint.html for details of this step.
  
guides/cvs_to_subversion.txt · Last modified: 2014/04/02 22:39 (external edit)