Cvs

From PTAGISWiki

CVS guide for common tasks 9/17/2004

To set up your environment to access the CVS repository

To determine if the environment is set properly, issue the command:

cvs -nq update

If you see a response like, "there is no version here; run 'cvs checkout' first", then you have it set up properly. If you see a response like, "No such file or directory", then cvs can't see the repository for some reason. Probably because the environment is not set. If you see "command not found", you need to add the path to CVS to your PATH environment variable, probably /usr/local/bin.

An additional environment variable needs to be set for remote access:

CVS_RSH=ssh

Remote access isn't fully set up in our environment yet. I'm still trying to find a way to accomodate Windows and Unix clients.

Unix client access to remote CVS repository

This is a method for extracting a copy (checking out) of a portion of the repository to a local machine. This allows changes to be made and tested in isolation before they are committed back to the repository.

cvs -d :ext:rday@pitblade:/dsk2/CVS checkout ptagis-1.0/web

After work is complete on this isolated copy of the repository, it can be discarded using

cvs -d :ext:rday@pitblade:/dsk2/CVS release -d ptagis-1.0/web

This command will warn you if you have changes that have not been applied and therefore will be lost by releasing the directory.

If you will be working with a particular checked out directory for some time, you will probably want to set your environment variable to make the external repository the default for all cvs commands like so:

set CVSROOT=:ext:rday@pitblade:/dsk2/CVS

To make a change to a file under CVS control

First, to determine if the file is under CVS management, issue the command

cvs status {filename}

If you get a report from CVS indicating the file's version number and status, then the file IS under CVS management.

Be sure you have the latest version of the file. It could be that someone else is working on this file and making changes to their own working copy of this file. Issue the command:

cvs update {filename}

If you get no output from this command, then you may start editting, knowing that you have the latest version of the file. If you get a line of output like this:

M gen_xml_news.sh

Then the local file and the CVS repository are out of sync and need to be reconciled before you can do your edits.

If the change is major enough to warrant a descriptive tag that will be kept in the history of the files, you should apply a CVS tag before the changes are commit. To apply the tag to the version of the files at the point of the last commit, use the command:

cvs tag USES_OLD_DB_TABLE gen_xml_news.sh

If you leave of the file name, the tag will be applied to all files.

After you make a change to the file, commit the change with this command:

cvs commit -m "Description of changes" <filename>

You should get a response like this:

Checking in gen_xml_news.sh;
/dsk2/ptagis-1.0/cvsArchive/ptagis/bin/gen_xml_news.sh,v  <--  gen_xml_news.sh
new revision: 1.3; previous revision: 1.2
done

Show files that have changed recently

The complete history of the repository can be seen with the command:

cvs history -ea -z PDT

This will show each change to each file in the entire repository. This is probably not what you want.

To narrow down the history report to just the files in a particular subdirectory, use this command

cvs history -ea -z PDT -p ptagis-1.0/web/ptagis/software/firmware

The -p parameter is the name of a project directory in the repository.

To narrow the history report by time of changes, use this command:

cvs history -ea -z PDT -D 04/13/05

The -D parameter tells CVS to display history records that are more recent than the specified date. The -D date parsing is pretty smart, so you can use dates like

cvs history -ea -z PDT -D "last week"

or

cvs history -ea -z PDT -D yesterday

Another option is to look at the CVS log to see what files in the current directory have entries within a given date range like this:

cvs -q log -S -R -d "04/18/05 < 04/19/05"

The -S parameter indicates that you only want to see entries that fall in the date range. The -R parameter indicates that you only want to see the name of the file that changed. So leave this off if you want to see the details of the change:

cvs -q log -S -d "04/18/05 < 04/19/05"

Or if you are lazy:

cvs -q log -S -d "yesterday<"

Or perhaps:

cvs -q log -S -R -d "1 hour ago<"

Delete a file

To delete a file use the cvs command:

cvs remove -f {filename}

This deletes the file in the current directory and marks the file deleted in the repository. The state of the file in the repository now looks like this:

[rday@snapper doc]$ cvs status deleteme.txt
===================================================================
File: no file deleteme.txt              Status: Up-to-date
	 
Working revision:    No entry for deleteme.txt
Repository revision: 1.2     /usr/local/cvs/doc/Attic/deleteme.txt,v

This says that as of version 1.2 the file is deleted. If we want to recover the file for any reason, we could just ask for the previous version from the repository like so:

cvs update -r1.1 deleteme.txt
U deleteme.txt

Now the file exists locally, but not in the repository. To bring the directory back to the state of the repository (deleting the file locally again), issue this command:

cvs update -A

How to add a binary file to CVS control

CVS has the default behavior of updating certain keywords within a file when it is checked in. For instance, a file might have a comment line like this:

// Current revision: $Revision$

When the file is checked in, the keyword $Revision$ is expanded to the actual revision of the file. This not the behavior you want if you are checking in a binary file that just happens to have that keyword in it.

To indicate a file is binary and should not be fiddled with when it is checked in, it should be added to the repository with this command:

cvs add -kb <file>

To add a directory of files to CVS control

If you are creating a new repository, create a directory to act as the repository and initialize it with this command:

cvs -d {repository directory} init

Set the environment variable for the CVS root. On reedi this is /home/ptagdev/CVS

export CVSROOT=/home/ptagdev/CVS

or

export CVSROOT=/home/ptagdev/CVScanary

cd to the directory, so that you are at the same level as the files you want to have in the repository.

cd wdir

cvs import -ko -I! -m "Imported sources" {logical name}/{wdir} ptagis start

This turns off keyword substitution "-ko"

• tells CVS not to ignore any files with -I!
• sets the initial import message with -m "Imported sources"
• assigns the module name
• assigns the vendor to ptagis
• and assigns the initial tag

For example, if the current directory is /dsk2/ptagis-1.0/src/util, then use this command to import all files there:

cvs import -ko -I! -m "Imported sources" ptagis-1.0/src/util ptagis start

(module name = {logical name}/{wdir}) The logical name component of the module name is necessary to prevent name clashes between directories with identical names but different paths. The logical name must not have a leading "/" since it is really a relative path within the CVS repository.

You should get a message about no conflicts being generated. cd up to the parent directory.

cd ..

This step can be skipped.

// Remove the original files.
//
//	rm -rf <wdir>

Checkout the files

cvs checkout -d {wdir} {logical name}/{wdir}

In our example, that would be:

cvs checkout -d util ptagis-1.0/src/util

Go back into the working directory:

cd util

Now the files are tracked in the repository and are present in filesystem. This is useful for tracking changes to config files or scripts that need to be visible to the system. Normally CVS-tracked files reside in the repository and only come out to be changed. So according to CVS, these files are always checked-out.

How can you put only certain files in a directory under CVS control?

For every file or directory that you do not want to be in the repository, create a regular expression to capture only that file name and make an entry in the file .cvsignore.

Place the .cvsignore file (or files, as necessary) in the target directory before you do the import as mentioned in the regular process for importing.

Can you put just a single file under CVS, and not a whole directory?

Only if that single file is in a directory by itself. CVS needs a directory to store its metadata. Alternately, you could create a .cvsignore file that lists every file except the single file you want to keep under CVS. But that just seems prone to failure as files come and go in the directory.

To see what files have changed since the last commit to the repository.

cvs -n -q update

That tells CVS to not update any files (-n) and be brief about the output (-q) but to pretend to do an update in the current directory.

To see the details of all the changes since the last commit.

cvs diff -u

To view the changelog for a file

cvs log {filename}

To view the file with annotations on each line indicating who made the change.

cvs annotate {filename}

How do you revert to a previous version of a file?

To get an idea of the changes made to the file and determine which version is the one you want to revert to, look at the log:

cvs log {filename}

For instance:

cvs log test_DFQ.pl

Displays this:

RCS file: /usr/local/cvs/ptagis/DFQ/test_DFQ.pl,v
Working file: test_DFQ.pl
head: 1.2
branch:
locks: strict
access list:
symbolic names:
       	start: 1.1.1.1
       	ptagis: 1.1.1
keyword substitution: kv
total revisions: 3;     selected revisions: 3
description:
----------------------------
revision 1.2
date: 2005/01/26 22:05:48;  author: rday;  state: Exp;  lines: +1 -1
made bogus change test -> temp
----------------------------
revision 1.1
date: 2004/09/17 22:23:30;  author: rday;  state: Exp;
branches:  1.1.1;
Initial revision
----------------------------
revision 1.1.1.1
date: 2004/09/17 22:23:30;  author: rday;  state: Exp;  lines: +0 -0
imported sources
=============================================================================

This file has 3 revisions and the change made on Jan 26 has a note indicating a bogus change made by user rday. Not all errors will be labeled so clearly, but this one is.

To get a clear idea of the changes made between revision 1.2 and 1.1, use this command:

cvs diff -r 1.1 -r 1.2 test_DFQ.pl

The output in this case is this:

Index: test_DFQ.pl
===================================================================
RCS file: /usr/local/cvs/ptagis/DFQ/test_DFQ.pl,v
retrieving revision 1.1
retrieving revision 1.2
diff -r1.1 -r1.2
3c3
< # test_DFQ.pl
---
> # temp_DFQ.pl

Only one line was changed.

That verifies that the version we want is 1.1. Use this command to revert to version 1.1:

cvs update -j1.2 -j1.1 test_DFQ.pl

Now the file in the sandbox has been rolled back to the state of version 1.1. Checking to see if we are in sync with the repository now will tell us that we don't have the latest version.

cvs diff -u

Here's the output:

cvs diff: Diffing .
Index: test_DFQ.pl
===================================================================
RCS file: /usr/local/cvs/ptagis/DFQ/test_DFQ.pl,v
retrieving revision 1.2
diff -u -r1.2 test_DFQ.pl
--- test_DFQ.pl 26 Jan 2005 22:05:48 -0000      1.2
+++ test_DFQ.pl 26 Jan 2005 22:19:09 -0000
@@ -1,6 +1,6 @@
#!/usr/local/bin/perl
#
-# temp_DFQ.pl
+# test_DFQ.pl
#
# Purpose: Verify proper operation of DFQ.pm
#

It tells us that the difference between the most recent version (1.2) and the version in the sandbox is the change from "temp_DFQ.pl" to "test_DFQ.pl". That's exactly the bugfix that we want to apply. To tell CVS that the version in the sandbox should be considered the latest version, just do a commit.

cvs commit -m "fixed name of program"

Now a look at the log has the entries we expect:

cvs log test_DFQ.pl

This should include these lines in the output:

description:
----------------------------
revision 1.3
date: 2005/01/26 22:26:01;  author: rday;  state: Exp;  lines: +1 -1
fixed name of program
----------------------------
revision 1.2
date: 2005/01/26 22:05:48;  author: rday;  state: Exp;  lines: +1 -1
made bogus change test -> temp
----------------------------

To view difference between versions of a file

cvs diff -r {version} -r {version} {filename}

Access the repository from a remote machine

The following environment variables need to be set up for remote access to work:

CVS_RSH=ssh
export CVS_RSH

Then, if you can ssh to the repository server, you can execute any cvs command.

Generate a patch of changes made to a module

From the top level working directory, create a patch file that describes all changes since the last commit.

cvs diff -c > patch-2004-09-22

That patch can then be applied using the command

patch < patch-2004-09-22

To revert to the state of the files at the point of the last commit

The cvs update command moves changes from the repository into the local sandbox. The cvs commit command moves changes to the repository from the local sandbox. Normally the update command preserves your local changes while getting changes from the repository, but it has an option (-C) which causes update to clear out local changes when getting files from the repository.

cvs update -C

This will bring the repository and sandbox in sync, renaming any differing files in the sandbox with a name like .#DFQ.pm.1.10.

You can also specify individual filenames that you want to revert back to the latest version in the repository.

To give the current set of files in a module a tag

In order to give a collection of files a common name, like ver_1_0 or pre-patch or SEF_7-0_SP3_customized, use the rtag command. For example, after importing the Sitescape installation into CVS, I want to tag it as "SEF_7_0_SP3_customized" in order to make it easy to do a diff between SP3 and SP4. The command to assign the tag is: rtag {tagname} {module} In this case, the command would be: rtag SEF_7-0_SP3_customized ryanwork/sitescape Note that valid characters for tag names are letters, numbers, hyphens, and underscores. No periods are allowed. In practice, use a hyphen where you would like to use a period and an underscore where you would use a space. This command has no effect on the files in the sandbox, but rather modifies the tag name of the files in the repository.

Steps to Sitescape installation under CVS on pitblade

1. cd /dsk2/local/sitescape
2. cvs -d /dsk2/CVS import -ko -I FrontBase -I hidden -I logs -I core -I

sockets -m "Imported sitescape customized source" local/sitescape ptagis start

1. cd ..
2. cvs -d /dsk2/CVS checkout -d sitescape local/sitescape

Then do a series of "cvs add *" and "cvs up" on each directory in the tree since checkout isn't recursive when the files still exist in the sandbox. Directories that should be ignored by CVS can be ignored by putting a .cvsignore file in them with the patterns that should be ignored in it. A pattern of "*" will ignore all files in the directory. When .cvsignore doesn't seem to be working, directories can be ignored by editting the CVS/Entries file to remove the D//* entries that correspond to the directory to be ignored. If a directory is already under CVS control and you put its parent under CVS control, things just work as you would hope. The whole tree is treated as one sandbox and files do not get duplicated.

Steps to put copy of Sitescape installation under CVS

On pitblade

1. cd /dsk2/installs/sitescape/src/
2. mkdir ryanwork
3. cd ryanwork
4. tar xvf site.tar
5. cd sitescape
6. cvs import -ko -I! -m "Imported sef-7.0 SP3 customized source"

ryanwork/sitescape ptagis start

1. cd ..
2. rm -rf sitescape/
3. cvs checkout -d sitescape ryanwork/sitescape
4. cvs -nq upd
5. cvs rtag SEF_7-0_SP3_custom ryanwork/sitescape

Apply SP4 patch

1. cvs -nq upd > SP3-SP4.diff

Determine list of files with customizations (grep for mods keyword) Determine union of set of customized files and SP3-SP4 diffs. In this case that list has 12 files and is stored in classes.custom For each file, reconcile the changes. Compare the version of the file from SP4 to the installed version. There should be two classes of differences: mods from SP4 mods from SAIC Make sure the SAIC mods are applied without colliding with SP4 mods. The following is a visual diff tool to make that comparison: meld /net/pitblade/dsk2/installs/sitescape/src/ryanwork/sitescape/classes/doc_customizations.module /net/pitblade/dsk2/local/sitescape/classes/doc_customizations.module

How to find all the directories under CVS control in a filesystem

CVS creates a directory called "CVS" in each directory that it manages, so it is as simple as this:

find . -type d -name CVS

How to commit after many files have been added

cvs -nq up marks the files that CVS doesn't know about with a ? Use this list to add all those files to cvs instead of issuing the command

cvs add dir/path/* 

for every directory with new files

cvs -nq upd | grep "^?" | cut -c3-99 | xargs cvs add

This will have to be repeated for as many new directories are added.

How to commit after many files have been deleted

cvs -nq up will report on files that have been deleted locally but still exist in the repository. These messages come on stderr instead of stdout, so special care has to be taken to capture the messages for use in a command that will remove them from the repository.

cvs -nq up 2>&1 | fgrep "was lost" | awk '{ print $4 }' | xargs cvs remove

The key is folding stderr into stdout with "2>&1". This works for the bash shell. Other methods may be required for other shells. This will fail if the file to be removed has a space in the name. To catch files with spaces in them use this instead:

cvs -nq up 2>&1 | fgrep "was lost" | \
nawk '{ sub(/cvs.*warning: /, "") ; print }' | \
nawk '{ sub(/ was lost/, "") ; print }' | \
xargs -i cvs remove "{}"

How to commit the SREE cvswd

SREE manages report versions with CVS, but it doesn't seem to manage it very well. Commits are left undone and files get lost. Occasionally this directory needs to be cleaned up.

The directory lives at /global/ds1/pitweb/ptagis-1.0/cvswd The CVS repository is at /global/ds1/pitweb/ptagis-1.0/cvsArchive

It would be good if you could just issue a normal commit and be done:

cvs commit -m ""

But the repository is often so far out of sync that manual intervention is required. To find out what needs to be done do the standard update check:

cvs -nq up

You'll probably get output like this:

R p4334536701117118566290.dat
cvs update: warning: p547363691118173352880.dat was lost
U p547363691118173352880.dat
cvs update: warning: p547363691118187704448.dat was lost
U p547363691118187704448.dat
cvs update: warning: p547373301118186848158.dat was lost
U p547373301118186848158.dat

For each file that is lost, you have to remove a line from the CVS/Entries file. First get a list of all those lost files like so:

cvs -nq up 2>&1 | grep "was lost" | nawk '{ sub(/cvs.*warning: /, "") ; \
print }' | nawk '{ sub(/ was lost/, "") ; print }' > ../lost.txt

Now do a reverse grep to find all the lines in CVS/Entries that do not contain one of those files:

fgrep -v -f ../lost.txt CVS/Entries

Do a word count to verify that everything is working as you would expect:

wc CVS/Entries
wc ../lost.txt
fgrep -v -f ../lost.txt CVS/Entries | wc

The number of lines in lost.txt plus the number of lines found by the reverse grep should add up to the number of lines in CVS/Entries.

Capture the output of the reverse grep and use it as the new CVS/Entries. Save the oldEntries and copy in the newEntries.

Issue another update check and hope that there are no more lost entries. If there are, they could have been generated during this process. Repeat as necessary.

Use the command:

cvs update -A

To recover all the report files that have deleted from the working directory without being deleted from the repository.

CVS puts reports that have been deleted from the archive into the 'Attic' so things never really get deleted. To actually free up disk space, the Attic should be emptied since these reports are no longer really visible to the application.

Alternately, you can use a different temp directory so as not to consume swap and impact other processes:

cvs -T /global/ds1/cache/tmp commit -m ""

A related procedure is to run the cvs-sync.pl script which compares the contents of the cvswd to the cvswd/cvsindex.xml. It generates a shell script which performs a "cvs remove -f" for each file that it finds that does not exist in the cvsindex.xml.

If, on commit, you get this error:

reedi.psmfc.org:C1:root: > cvs commit -m "" cvs commit: Examining . cvs commit: Up-to-date check failed for `p10167093171114044012982.dat'

Issue a "cvs up" and try again.

After working this way with CVS, make sure that there are no temporary files chewing up lots of swap space. CVS may leave files in /tmp that begin with "cvs":

ls -lt /tmp/cvs*

How to capture the diffs from development and apply to production

You want to capture the changes made on pitblade in a given directory since the last commit. You can give a high-level summary of those changes with the familiar command:

cvs -nq up

You can also get a complete description of those changes by capturing the output of this command:

cvs -q diff -u > /tmp/aug18.diff

Then move the diff file to production and cd to the directory where you want to apply the changes. The command to apply the diffs is this:

patch < aug18.diff

This procedure will not work with binary files. And patch will fail on text files that don't have an ending newline. Such files will just have to be copied from development to production.

How to remove a file with a sticky tag

A sticky tag is an indication that a file has been checked out as a static copy that cannot be altered. This is useful for looking at past versions. But a file with a sticky tag cannot be removed or checked in to the repository. First, the tag must be cleared from the CVS/Entries file with this command:

cvs update -A {file}

Then the file can be removed normally:

cvs remove -f {file}

How to create a tar package of changed files

Create a temporary file listing each changed file, one per line.

  cvs -nq up | awk '{print $NF }' > temp.txt

Use that temporary file as input to tar.

   tar cvf new.tar -I temp.txt