What are the appropriate continuous integration retention policies?

Share URL
 
Per Lundborg's picture
Per Lundborg asked on April 15, 2011 - 8:40am | Replies (5).

I am working at a 500 SW developer shop as a SCM Tool specialist. Our products are under supervision from legal standards and authorities (very much like the medical industry with FDA). Within the SCM tool we store source code, built items and product documentation.

The problem at hand is that we are coming up against a limit to how much data we can host, as we currently do not have an active retention policy, and I am asked to come up with one.

It would be interesting to learn of how other businesses are dealing with the deletion of CI's (that are between baselines). Best practices?

Stories from our kind of business is a bonus, but any feedback is most welcome too.

The problem might only exist for certain SCM tools, all depending on how CI's are stored, but it is certainly a problem for us.

Tags: 
configuration management
ci retention

5 Answers

jptownsend's picture

PeL,

Well it all depends on the Products life and how long it will be used and how long you are legally responsible for its safety. You never gave specifics to what industry you are in.

Without knowing specifics, I would say keep the data forever. Two reasons, space is super cheap now. You can get hundreds of terabytes of space for almost nothing, you can burn the data to discs, platters, etc. The second reason, legally when data is not available and it is proven you got rid of it or deleted it, it is the same as saying we are guilty that is why we got rid of it.

So just to be safe, keep it and keep all of it, remember you can archive files, hard drives, platters, etc a whole lot cheaper than you can defend a lawsuit, especially when the evidence that would have proven your innocence has been deleted.

Regards,

Joe

davidbfoster's picture

PeL,
I agree with Joe. You did not identify the industry but as Joe pointed out, storage is one heck of a lot cheaper than a lawsuit. Does your employer have any type of data retention policy in place?
Good Luck,
Dave

Per Lundborg's picture

Thanks all of you!

sorry for omitting the technology area (while asking for examples from "our" area) - which in our case is in railway supervision. The product lifespan can ba as long as 30 years. In some contracts that is the functional lifespan (ie we might choose to replace the original product with one having Form-Fit-Function compabilities), in other we are more bound the the actual product.

My main concern here were not those released things that actually makes it to the customer, those are undeniable subject for long term storage and availablilty, what I was aiming at was all those bits and piecees that are created along the way, all those interim versions between baselines.

I will take Bill's considerations at the end of his entry onboard and see how to develop from there. He clearly reflects the thoughta of the more cautions members here at my site. The bottom line seems to be a bit of "we don't know until we have been in a lawsuite, and found out what we needed, and then finding the cost of not having found what we should have"... Certainly not a situation that any senior manager wants to find h*self in.

BR
/PeL

bglangston's picture

Thanks for the consideration. To paraphrase one of the very old mantras that especially addresses the cost of CM, "You can pay a little now or a lot later, but you're gonna pay." That's not only in very practical terms of money, but also in the bureaucratic nightmares.

Since you've clarified the product somewhat, I'll add that in such situations I also strive to have Advisory Members of the decision making body (CCB). There are often situations or proposals that require assessment and feedback from such entities as legal, finance, security and even HR. These people, in my designs, usually don't have voting rights on the "best technical solution." They do have often have very important inputs to the final decision maker and including their remarks on the proposal form keeps it all in a tidy place. Of course, that arrangement means that the CCB doesn't have the final decision on the "go-ahead," rather the final decision on what is the best technical solution.

lancelyons's picture

I am also in agreement that you keep as much as you can and expand your SVN repository if possible. Here is what we do at our company.

source code is retained for the life of a product including all tags or version labels.

For release code (builds) - we keep all builds between the current release we are working on and the last release after that we only keep the builds that were released.

Product documentation is similar to source code in that all prior release sets of documentation should be retained and typically this does not cause a problem since documentation will normally not take much space.

Since Subversion has no clean way of deleting artifacts permanently from the repository, we periodically reduce the size of our built releases repository using svnadmin dump, svndumpfilter to filter out old release builds and then svnadmin load.

Sometimes this isn't practical you you must recreate the repo from scratch bringing in only the items you want to retain. You would want to backup the original built item repository before removing completely.

CMCrossroads is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.