Contents
Replicating an existing repository
On the one hand SVNKit ver. 1.1 supports the Subversion ver. 1.4 feature of synchronizing two repositories. On a low-level the SVNRepository class provides a user replay() method which accepts a client's commit editor to commit all changes made in a particular revision of the source repository to a target one:
1 public abstract void replay( long lowRevision , long revision , boolean sendDeltas , ISVNEditor editor ) throws SVNException;
SVNKit ver. 1.1 also brings a high-level API to do synchronization just as the Subversion's svnsync utility does. SVNAdminClient provides such API methods as
doInitialize(), similar to svnsync initialize command
doSynchronize(), similar to svnsync synchronize command
doCopyRevisionProperties(), similar to svnsync copy-revprops command
For repositories accessible through the file:/// protocol or running under servers which support the replay functionality, this way of synchronizing two repositories should work well. But what if we would like to replicate a repository that is running under Apache or svnserve which don't support this functionality. In this case SVNKit provides its own feature that can replicate repositories running under older servers - repository replicator.
The repository replicator is a tool that gives you an ability to create clones of existing repositories. That is, the replicator copies revisions of existing repositories to other clean repositories what results in two separate repositories containing the same versioned data.
The replicator is represented by the class SVNRepositoryReplicator. Here's a lite class diagram demonstrating relationships between the replicator and other components which are involved by the replicator:
Repository replicator: replicating a range of source repository revisions
There are two ways you can replicate repositories. The first one corresponds to using the replicator's
1 public long replicateRepository( SVNRepository src , SVNRepository dst , long fromRevision , long toRevision ) throws SVNException
method.
Both source (src) and destination (dst) SVNRepository drivers must be created for the root locations of the source and destination repositories respectively. That is, only entire repository trees can be replicated, but not sub-trees. A destination repository must be either completely clean (be at revision 0) or must already contain all source repository revisions in the range starting from revision 1 and up to fromRevision - 1 inclusively. In all cases, if the destination repository latest revision is not equal to fromRevision - 1, you will get an exception.
If fromRevision is less or equal to 0, it automatically defaults to revision 1, since revision 0 is always a point of repository life start and doesn't contain any user valuable data. In all cases when toRevision is not in this range - toRevision > 0 and toRevision < source repository latest revision - it automatically defaults to the source repository latest revision.
Repository replicator: Replicating a source repository incrementally
This way of replicator usage corresponds to using the replicator's
1 public long replicateRepository( SVNRepository src , SVNRepository dst , boolean incremental ) throws SVNException
method.
Incrementally means that sometime you replicated the source repository into the destination one, and since then have committed some more revisions into the source repository. Or maybe that time you replicated not the entire range of the source repository revisions. And in case you would also like to copy those revisions to the destination repository, you perform an incremental copy - copy those missing revisions. In fact, this method is equivalent to
That is, if you set incremental to false the whole source repository is replicated. Otherwise the source repository is incrementally copied up to the latest revision.
Repository replicator: replicating principles
Starting with the first revision to copy the replicator obtains information about all changed paths in the particular revision of the source repository. This information is analysed for the presence of paths copied in that revision. Then the replicator obtains a [:Committing To A Repository:commit editor] for the destination repository. This editor is passed to an instance of SVNReplicationEditor. A replication editor is like a bridge between the source and destination repositories: the replicator calls an update operation for the same particular revision on the source driver passing it a replication editor as an update editor. When the source driver calls the replication editor, the latter transforms these calls to calls to the commit editor of the destination driver. Thus changes received are redirected immediately to the destination repository.
SVNReplicationHandler is introduced to control replication process. At the start of each replication iteration the replicator checks the registered handler's method
1 public void checkCancelled( ) throws SVNCancelException
To stop the replication process it's enough for the handler to throw an SVNCancelException exception. replicateRepository(...) method will throw it upper.
If replication is not cancelled and a log operation is successfully perfromed for a particular revision of the source repository, the replicator passes this log info to the registered handler's method
1 public void revisionReplicating( SVNRepositoryReplicator source , SVNLogEntry logEntry ) throws SVNException
And after the current revision is successfully committed to the destination repository the replicaor passes commit info to the handler's method:
1 public void revisionReplicated( SVNRepositoryReplicator source , SVNCommitInfo commitInfo ) throws SVNException;
The replicator copies both versioned and unversioned (revision properties) data.
In theory, with the repository replicator you are able to create exact copies of repository contents, but in practice it strongly depends on permissions you have on both source and destination repositories. Say, if you are not able to read some directory in the source repository, it is obvious that this directory will be missing in the destination one.
You can use different repository access [:SVNKit Architecture:protocols supported by SVNKit] for both source and destination repositories. This provides you an ability to make a local back-up of a repository located on the server machine.
Example: synchronizing/replicating a repository
In this example we will replicate a source repository to a local destination one. Our program should be able to receive two args: first one is the url of the source repository and the second one is the path of the target repository. If no args are provided, we create a local FSFS-type repository and fill it with some data. Here's a function which populates a repository with some dummy data.
1 private static void populateSourceRepository( SVNRepository srcRepository ) throws SVNException {
2 /*
3 * Simple repository tree to create. Each entry will be
4 * added in its own revision.
5 */
6 String dirA = "dirA";
7 String dirB = "dirA/dirB";
8 String fileA = "dirA/fileA.txt";
9 String fileB = "dirA/dirB/fileB.txt";
10 byte[] fileAContents = "This is file fileA.txt".getBytes( );
11 byte[] fileBContents = "This is file fileB.txt".getBytes( );
12 SVNDeltaGenerator deltaGenerator = new SVNDeltaGenerator( );
13 long revision = -1;
14 SVNCommitInfo info = null;
15 String checksum = null;
16
17 /*
18 * First commit "/dirA".
19 */
20 ISVNEditor editor = srcRepository.getCommitEditor( "adding " + dirA , null );
21 editor.openRoot( -1 );
22 editor.addDir( dirA , null , -1 );
23 editor.closeDir( );
24 editor.closeDir( );
25 info = editor.closeEdit( );
26 System.out.println( info );
27 revision = info.getNewRevision( );
28
29 /*
30 * Then commit "/dirA/fileA.txt".
31 */
32 editor = srcRepository.getCommitEditor( "adding " + fileA , null );
33 editor.openRoot( -1 );
34 editor.openDir( dirA , revision );
35 editor.addFile( fileA , null , -1 );
36 editor.applyTextDelta( fileA , null );
37 checksum = deltaGenerator.sendDelta( fileA , new ByteArrayInputStream( fileAContents ) , editor , true );
38 editor.closeFile( fileA , checksum );
39 editor.closeDir( );
40 editor.closeDir( );
41 info = editor.closeEdit( );
42 System.out.println( info );
43 revision = info.getNewRevision( );
44
45 /*
46 * Then commit "/dirA/dirB".
47 */
48 editor = srcRepository.getCommitEditor( "adding " + dirB , null );
49 editor.openRoot( -1 );
50 editor.openDir( dirA , revision );
51 editor.addDir( dirB , null , -1 );
52 editor.closeDir( );
53 editor.closeDir( );
54 editor.closeDir( );
55 info = editor.closeEdit( );
56 System.out.println( info );
57 revision = info.getNewRevision( );
58
59 /*
60 * Then commit "/dirA/dirB/fileB.txt".
61 */
62 editor = srcRepository.getCommitEditor( "adding " + fileB , null );
63 editor.openRoot( -1 );
64 editor.openDir( dirA , revision );
65 editor.openDir( dirB , revision );
66 editor.addFile( fileB , null , -1 );
67 editor.applyTextDelta( fileB , null );
68 checksum = deltaGenerator.sendDelta( fileB , new ByteArrayInputStream( fileBContents ) , editor , true );
69 editor.closeFile( fileB , checksum );
70 editor.closeDir( );
71 editor.closeDir( );
72 editor.closeDir( );
73 info = editor.closeEdit( );
74 System.out.println( info );
75 }
However if args are provided we must do some checks:
if the source url is a file:// url and there's no repository at this place, we also create a new one and fill it up with some data
if the source url is a file:// url and there's already a repository at this place and its latest revision is 0, we fill it with some data
in all other cases (non file:// access and empty repositories accessible through file://) we use the data from the source repository
1 public class Replicate {
2
3 public static void main( String[] args ) {
4 /*
5 * Default values:
6 * source and target repository paths
7 */
8 String srcPath = "srcRepository";
9 String tgtPath = "tgtRepository";
10 String srcUrl = null;
11 /*
12 * Initializes the library (it must be done before ever using the
13 * library itself)
14 */
15 setupLibrary( );
16
17 if ( args != null ) {
18 /*
19 * a source repository url
20 */
21 srcUrl = ( args.length >= 1 ) ? args[0] : srcUrl;
22 /*
23 * a target repository path
24 */
25 tgtPath = ( args.length >= 2 ) ? args[1] : tgtPath;
26 }
27
28 SVNURL srcURL = null;
29 SVNURL tgtURL = null;
30 SVNRepository srcRepository = null;
31 SVNRepository tgtRepository = null;
32 boolean createSrcRepos = false;
33 boolean populateSrcRepos = false;
34 try {
35 if ( srcUrl != null ) {
36 /*
37 * If a source url was provided, using it as a source repository
38 */
39 srcURL = SVNURL.parseURIDecoded( srcUrl );
40 if ( "file".equals( srcURL.getProtocol( ) ) ) {
41 File srcReposDir = new File( srcURL.getPath( ) );
42 if ( !srcReposDir.exists( ) ) {
43 /*
44 * it's a local access scheme and src path does not exist -
45 * we'll need to create something
46 */
47 createSrcRepos = true;
48 populateSrcRepos = true;
49 srcPath = srcURL.getPath( );
50 } else {
51 srcRepository = SVNRepositoryFactory.create( srcURL );
52 if ( srcRepository.getLatestRevision( ) == 0 ) {
53 /*
54 * it's a local access scheme, src path already
55 * exists, but seems to be an empty repository -
56 * we'll need to create something in it
57 */
58 populateSrcRepos = true;
59 }
60 }
61 }
62 } else {
63 createSrcRepos = true;
64 populateSrcRepos = true;
65 }
66
67 if ( createSrcRepos ) {
68 srcURL = SVNRepositoryFactory.createLocalRepository( new File( srcPath ) , false , false );
69 }
70
71 /*
72 * For the target repository we need to enable revision property
73 * changes.
74 */
75 tgtURL = SVNRepositoryFactory.createLocalRepository(new File( tgtPath ) , true , false );
76
77 srcRepository = SVNRepositoryFactory.create( srcURL );
78 tgtRepository = SVNRepositoryFactory.create( tgtURL );
79 } catch ( SVNException svne ) {
80 /*
81 * getFullMessage() will give us a full tree of SVNException
82 * errors.
83 */
84 System.err.println( "Can not create a repository: " + svne.getErrorMessage( ).getFullMessage( ) );
85 System.exit( 1 );
86 }
87
88 /*
89 * Deault auth manager is used to cache a username in the
90 * default Subversion credentials storage area.
91 */
92 srcRepository.setAuthenticationManager( SVNWCUtil.createDefaultAuthenticationManager( ) );
93 tgtRepository.setAuthenticationManager( SVNWCUtil.createDefaultAuthenticationManager( ) );
94
95 try{
96 if ( populateSrcRepos ) {
97 /*
98 * Fills up the source repository with some data.
99 */
100 populateSourceRepository( srcRepository );
101 }
102 System.out.println( );
103 long srcLatestRevision = srcRepository.getLatestRevision( );
104 System.out.println( "The latest source revision: " + srcLatestRevision );
105 System.out.println( );
106 System.out.println( "'" + srcURL + "' repository tree:" );
107 System.out.println( );
108 /*
109 * Using the DisplayRepositoryTree example to show the source
110 * repository tree in the latest revision.
111 */
112 DisplayRepositoryTree.listEntries( srcRepository , "" );
113 System.out.println( );
114 } catch( SVNException svne ) {
115 System.err.println( "An error occurred while accessing source repository: " + svne.getErrorMessage( ).getFullMessage( ) );
116 System.exit( 1 );
117 }
118 ...
First, we will try replay() functionality for replicating the source repository. We'll use a simple initialization function that copies revision props from revision 0 of the source repository to the same revision of the target one:
1 private static void initializeRepository( SVNRepository fromRepos , SVNRepository toRepos ) throws SVNException {
2 /*
3 * Initialization means we need to set necessary svn:sync- properties
4 * on revision 0 of the destination repository. But since our program
5 * is just an example, we copy only revision properties from the
6 * source repository to the destination one.
7 */
8 copyRevisionProperties( fromRepos , toRepos , 0 );
9 }
The routine for copying revision properties:
1 private static void copyRevisionProperties( SVNRepository fromRepository , SVNRepository toRepository , long revision ) throws SVNException {
2 Map revProps = fromRepository.getRevisionProperties( revision , null );
3 for ( Iterator propNames = revProps.keySet( ).iterator( ); propNames.hasNext( ); ) {
4 String propName = ( String ) propNames.next( );
5 String propValue = ( String ) revProps.get( propName );
6 toRepository.setRevisionPropertyValue( revision , propName , propValue );
7 }
8 }
And synchronization itself:
1 private static long synchronizeRepository( SVNRepository fromRepos , SVNRepository toRepos ) throws SVNException {
2 long lastMergedRevision = 0;
3 long fromLatestRevision = fromRepos.getLatestRevision( );
4 long count = 0;
5 for ( long currentRev = lastMergedRevision + 1 ; currentRev <= fromLatestRevision ; currentRev++ ) {
6 ISVNEditor commitEditor = toRepos.getCommitEditor( "" , null );
7 fromRepos.replay( 0 , currentRev , true , commitEditor );
8 SVNCommitInfo info = commitEditor.closeEdit( );
9
10 if ( info.getNewRevision( ) != currentRev ) {
11 System.err.println( "Commit created rev " + info.getNewRevision( ) + " but should have created " + currentRev );
12 System.exit( 1 );
13 }
14 System.out.println( "Committed revision " + info.getNewRevision( ) );
15 copyRevisionProperties( fromRepos , toRepos , currentRev );
16 count++;
17 }
18 return count;
19 }
If we can't replicate the source repository in case a server does not support replay functionality we would like to use SVNRepositoryReplicator:
1 private static long replicateRepository( SVNRepository srcRepository , SVNRepository tgtRepository ) throws SVNException {
2 long latestRevision = srcRepository.getLatestRevision( );
3 SVNRepositoryReplicator replicator = SVNRepositoryReplicator.newInstance( );
4 //use this handler to print out progress information on commits
5 replicator.setReplicationHandler( new ISVNReplicationHandler( ) {
6
7 public void revisionReplicated( SVNRepositoryReplicator source , SVNCommitInfo commitInfo ) throws SVNException {
8 System.out.println( "Committed revision " + commitInfo.getNewRevision( ) );
9 }
10
11 public void revisionReplicating( SVNRepositoryReplicator source , SVNLogEntry logEntry ) throws SVNException {
12 }
13
14 public void checkCancelled( ) throws SVNCancelException {
15 }
16 });
17
18 return replicator.replicateRepository( srcRepository , tgtRepository , 1 , latestRevision );
19 }
In the main program we call this routines:
1 ...
2
3 try{
4 /*
5 * First let's try the standard replay way.
6 */
7 long replicatedRevisions = 0;
8 try {
9 initializeRepository( srcRepository , tgtRepository );
10 replicatedRevisions = synchronizeRepository( srcRepository , tgtRepository );
11 } catch ( SVNException svne ) {
12 if ( svne.getErrorMessage( ).getErrorCode( ) != SVNErrorCode.RA_NOT_IMPLEMENTED ) {
13 throw svne;
14 }
15
16 //...else the server does not support replay functionality
17 tgtRepository = SVNRepositoryFactory.create( tgtURL );
18 replicatedRevisions = replicateRepository( srcRepository , tgtRepository );
19 }
20
21 ...
If the server does not support replay functionality we get SVNErrorCode.RA_NOT_IMPLEMENTED and invoke our replicator. tgtRepository is reinstantiated since it may be locked by its getCommitEditor() method. At the end we print out the number of replicated revisions and the target repository tree:
1
2 ...
3
4 System.out.println( );
5 System.out.println( "Number of replicated revisions: " + replicatedRevisions );
6 System.out.println( );
7 System.out.println( "'" + tgtURL + "' repository tree:" );
8 System.out.println( );
9 /*
10 * Shows the tree of the target repository in the latest revision.
11 */
12 DisplayRepositoryTree.listEntries( tgtRepository , "" );
13 }catch( SVNException svne ) {
14 System.err.println( "An error occurred while accessing source repository: " + svne.getErrorMessage( ).getFullMessage( ) );
15 System.exit( 1 );
16 }
17 }
18 }
Running a program:
r1 by 'me' at Wed Apr 26 02:10:14 NOVST 2006 r2 by 'me' at Wed Apr 26 02:10:14 NOVST 2006 r3 by 'me' at Wed Apr 26 02:10:15 NOVST 2006 r4 by 'me' at Wed Apr 26 02:10:15 NOVST 2006 The latest source revision: 4 'file:///G:/tgtRepository' repository tree: /dirA (author: 'me'; revision: 4; date: Wed Apr 26 02:10:15 NOVST 2006) /dirA/dirB (author: 'me'; revision: 4; date: Wed Apr 26 02:10:15 NOVST 2006) /dirA/dirB/fileB.txt (author: 'me'; revision: 4; date: Wed Apr 26 02:10:15 NOVST 2006) /dirA/fileA.txt (author: 'me'; revision: 2; date: Wed Apr 26 02:10:14 NOVST 2006) Committed revision 1 Committed revision 2 Committed revision 3 Committed revision 4 Number of replicated revisions: 4 'file:///G:/tgtRepository' repository tree: /dirA (author: 'me'; revision: 4; date: Wed Apr 26 02:10:15 NOVST 2006) /dirA/dirB (author: 'me'; revision: 4; date: Wed Apr 26 02:10:15 NOVST 2006) /dirA/dirB/fileB.txt (author: 'me'; revision: 4; date: Wed Apr 26 02:10:15 NOVST 2006) /dirA/fileA.txt (author: 'me'; revision: 2; date: Wed Apr 26 02:10:14 NOVST 2006)
Download the example program source code.