devops Questions

Am working in a old project which has many hg subrepos which seems to be problematic. Am planning to remove subrepos and move to main repository. Is there any better approach to do the same or any tools? Also want to maintain the same metadata and not interested to add all subrepo as single commit to main repository.

Dear Experts,

Need your expter advice in setting up a new repository structure for our product development.

We have a software product, with 15-compnents and each of the 15-components have at least 15-modules.

Earlier our admin guy had configured the Stash server for us.

The current setup is like this:
Our IT engineer had created 15-Projects in stash/bitbucket server for 15-Components
For each of the 15-modules in a given component, a individual repository was created

Thus we have around 225 + 25(for additional module), totally 250 repositories to manage.

Individual developers creates their feature branches on their respective repositories and merge them to their production branch in their respective repo, after release.

Feature branches gets created only on a particular module in a particular repository which is getting affected for that feature.

If a developer is working on a feature, which is affecting 10-modules, this developer creates a branch in each of the repository which is affectting this module. Thus he creates 10-branches for a given feature in 10-different repositories.

In this way, we have many small teams working on many individual repositories corresponding to their module. Packages are made from each of the individual repository and delivered to our infra team, who deploys the product on our hosted servers.

Since source is spread all over the place, there is no baselines/tags created for this product. Also managing these many repositories by a single CM admin is too much to handle.

As a CM admin, I’m thinking of suggesting this structure for this product.
Create a single Project (Project A) in Stash 
Create a single repository (Repo A ) in this project 
Create a folder for each Component, i.e 15-Folders, under the repository root 
(i.e Project A - /RepoA/Component1/Module1 Module2 ……Module15
/RepoA/Component2/Module1 Module2 ……Module15
/RepoA/Component3/Module1 Module2 ……Module15
:
:
/RepoA/Folder15/Module1 Module2 ……Module15)

In this way we’ll have single repository to handle and CM admin will create branches and developers will just make changes and commit their code.

Appreciate your response on these lines:

1. Is my suggestion to host a single repository for all the components correct? 
is this achievable, what is your opinion on this repository structure?

2. We are developing a multi-tier, web application using PHP, Python, and Java Script. This application gets deployed on multiple servers. By using this single repository structure, do you see any issue or obstacle at a later stage on the line of development? What precautions should we take, if any?

3. Since the application is multi-tier, developers actually develop the application in the development environment (where multiple servers are setup to create a production sort of environment) using the shared work area in the environment, at times there are chances of one developer overwriting the others' changes.

What is your opinion on this kind of development?

4. Using this repository structure, how do we resolve the conflicts? Whenever a developer tries to push/pull the changes, he/she might face multiple conflicts not only from their changes. Should we call all developers to assemble at one desk and resolve the conflicts or is there any other better way?

5. We have many set of features getting developed on different branches. At times, many components are not modified in the some branches, still we just package them and release. What is your advice on this?

As CM admin, I see lots of advantages in managing branches and merges in a single repository structure. What do you suggest for our kind of development.

I have read many articles on web about the advantages/disadvantages of having single/multiple repositories. My Dev team is not conveienced with my approach.  Thus I need answers to all my(actually their's) question in one place.

Would appreciate your detailed response for each of the above issues that we are facing.  Your assistance in this regard will be very much appreciated.

Eagerly awaiting your response.

Thanking you,

Deepak.

Hello All ,

I am looking for a solution on how can we monitor the traffic received on UDP ports ( preferably through Nagios)

As per our setup , we are using Data Diode for highly secure unidirectional flow . There is a backend third party application which is sending data and it comes through data diode to the top layer on specific UDP port ( dual node active-active setup for high availability ) .

Now sometimes we fall in the situation when one of the data diode leg stop working which ultimately impacts the overall  data flow .

As per current situation , I am using a customised shell script which uses TCPDUMP on UDP ports to ensure we are receiving data on both leg of Data daiode . The script use to monitor the TCPDUMP traffic every 15 min and if no data received in last 15 min then sleep for 10 min and check again.

If no data received while second check then raise an alarm via Nagios .

Please let me know if we can use more optimised solution to monitor this setup .

Thanks in Advance !!

How have others solved the resourcing justification for Build / Release engineers? Based on number of supported developers? number of tools? number of deployments? are there base line ratios or other industry standards based on specific products or supported platforms?