Jump to content
The Dark Mod Forums

Using git


Alberto Salvia Novella

Recommended Posts

I strongly suggest that The Dark Mod source code is stored on a git based platform. Git makes contributions notably easier than SVN for the following reasons:

  • Git is able to merge commits with minimum conflict and with great transparency.
  • Git is distributed. Everyone has its own version of the source code, and different teams can be working in parallel on different features without needing to commit to the main repository, only to their copy till the feature is truly stable.
  • The different applications can be stored in different repositories, so cloning them and committing is fast.
  • These mean that changing things is easy to anybody without making the code unstable. This naturally encourages contribution and enhances overall software quality.
  • Since there's multiple copies of the source repository, and it's stored on an external platform, the likelihood of data lost is dramatically reduced.

Long version:

 

Edited by Alberto Salvia Novella
Link to comment
Share on other sites

@Springheel Cause it was never well designed to start with, having to commit to a single central repository. People used it only because it was the sole thing available at the time.

As everything is committed to a single central place:

  1. Anything can break the main code, so micro-managing commits is needed.
  2. Code conflicts can easily happen.
  3. You have to clone every related software for minor changes.
  4. So people delay commits, making them bigger, harder to inspect, and more prone to errors.
  5. And developers distrust outsiders by default, so these don't even try.

For the record most of the code I write is Unix Shell, invented like 40 years ago when machines used typewriters instead of screens. Don't tell me I have new shinny technology syndrome.

Link to comment
Share on other sites

First, please stop lecturing people on the internet, especially in this forum. You'll just generate posts dripping with sarcasm.

Secondly, to address the actual point: do you honestly think the thought of switching to Git has never occurred to the team before?

Link to comment
Share on other sites

Going in circles is fun, so let me reiterate my desire to move the source code to github (or similar) and VCS to git

SVN is great for a single developer going slowly from version N to N+1 doing planned changes

Not so much for community collaborations spreading across many areas and unrelated to each other. Especially when some of the contributors are known to break things every now and then.

Link to comment
Share on other sites

Holy war mode on ?

Quote

But literally all the software I know of no longer use SVN, they all migrated to Git.

You probably know little software, or only a very limited class of software. See Myth #5.

Quote

The different applications can be stored in different repositories, so cloning them and committing is fast.

Keeping interconnected applications in single monolithic repo has a lot of benefits over having different repositories. The main benefit is that every revision is set in stone and perfectly reproducible afterwards. It is also possible to do atomic changes to several projects. Just read about why Google does that. SVN allows you to checkout only a few subdirectories of the repo, so monolithic repo does not forbid people to download only what they need. Git sort-of has sparse checkouts, but there are complicated, most likely limited, and etc. Submodules... I'd better never know of their existence ?

Quote

Code conflicts can easily happen.

Yes, SVN is known for having more conflicts and having worse tools for resolving them. But with trunk-based development, they rarely happen. On the other hand, conflicts are the core nature of all VCS we use. Don't tell tales that conflicts won't happen at all with git.

Quote

So people delay commits, making them bigger, harder to inspect, and more prone to errors.

The same thing happens with git when people work via merge requests. Which is how it is usually done, especially for outer contributors.

Quote

Especially when some of the contributors are known to break things every now and then.

Changing VCS won't help you much with not breaking trunk. Unless you are going to work in your private branch for months without merging back, in which case nobody will ever review&accept your merge request due its gigantic size :D

Quote

Everyone has its own version of the source code, and different teams can be working in parallel on different features without needing to commit to the main repository, only to their copy till the feature is truly stable

Yes, and this is very dangerous, because most of developers are poor testers. They will gather bugs in their branches, and then when they consider them "truly stable", merge all these bugs into trunk/develop. And nobody will notice it soon, since everyone is deep in his own branches, branched from develop a month ago.

I prefer following the most widely used practice of game development, when everyone is sitting on trunk, so that all bugs get noticed and fixed quickly. Do you know that in Naughty Dogs artists even have an auto-update script for assets repo? So that when someone commits something, everybody forcibly gets the updated version.

Quote

the likelihood of data lost is dramatically reduced.

This is a relevant topic due to recent server outages. It is planned that our admin will provide readonly access to current backups to several members, to be sure that the whole TDM knowledge base does not suddenly disappear.

Speaking of normal usage, SVN has zero chance of losing data, since there is no history rewrite. Git, on the other hand, has lots of unsafe ways to edit history, and it is often encouraged. Which provides a good opportunity for users to lose their local history. And in worst case, I guess even remote history can be lost. Recall that git is not a Version Control System, it is a "Stupid Content Tracker". Stupid enough to lose the content it tracks.

Holy war off?

 

I think the main reasons why SVN stays on TDM are:

  1. Aside from the engine source code repo, we have a few more repos. They contain assets, and migrating them to git would be a disaster. And since they stay on SVN, source code should better stay on SVN too. Avoiding one more VCS is enough reason in itself.
  2. Git is inherently harder to use than SVN. Moreover, it is harder not because it has to be so, but because it was ill-designed. I don't like the idea of forcing complexity onto everyone (including myself BTW). Whoever wants to use git can take the additional complexity onto himself and use git-svn, or any other approach to synchronize SVN with git. Also, we have artists: while usually they don't work with source code repo, sometimes they decide to build the engine themselves.

There are ways to use git locally with SVN. I do this sort of thing myself with hg.

Some time ago I suggested creating an official GitHub mirror for source code repo, with some semi-automatic workflow for accepting pull requests. As far as I remember, nobody was interested. And migration is unlikely to happen.

 

  • Like 1
Link to comment
Share on other sites

I'm a big fan of Git. I switched DarkRadiant to Git years ago, and have been happily using it and sharing commits between my and Greebo's repositories (on entirely separate hosting platforms) since then. But even as a Git fanboy I am embarrassed by some of the misconceptions about, both in the "for" and "against" camps.

20 hours ago, Alberto Salvia Novella said:

As everything is committed to a single central place:

Misconception: "Git doesn't use a central server".

Reality: The vast majority of Git projects, and the workflow that would most likely be adopted by TDM if it were to switch to Git, is to have a central authoritative server that every trusted developer pushes their changes to. The Linux kernel is a notable exception, but is also very atypical in both its size and the distributed nature of its development. Git provides a little more flexibility in that developers can make a whole string of commits locally and then push them in one go, but they are still going to be pushed to that central server before anyone else can use them. Having each individual developer push their changes to their own GitHub repository and submit a pull request would be possible, but probably wouldn't solve any problems for this project, and would increase workload for whoever is appointed to review and accept all of the pull requests.

Quote
  1. Anything can break the main code, so micro-managing commits is needed.

Since all developers would be pushing to the same repository, code breakage is just as possible with Git also.

Quote
  1. Code conflicts can easily happen.

Misconception: "Git magically avoids code conflicts".

Reality: Code conflicts are the result of divergent, incompatible text changes in the code itself. If you and I edit the same line of code and then try to push our changes, a merge conflict will happen. The choice of VCS makes absolutely no difference to whether this conflict will occur, although Git does provide slightly better tools for dealing with the conflict when it does arrive.

Quote
  1. You have to clone every related software for minor changes.

There are only two repositories that matter for the Dark Mod: the source code repository, and the assets repository (which is never going to be switched to Git because it is a repository of large binary files which would not be handled well by any distributed VCS tool). So needing to clone the whole thing is not an issue.

Quote
  1. So people delay commits, making them bigger, harder to inspect, and more prone to errors.
  2. And developers distrust outsiders by default, so these don't even try.

Those are both workflow or cultural issues which cannot be solved by a VCS. People can just as easily delay commits/pushes with Git, and they can easily make lots of small incremental commits using Subversion. If commit frequency is a problem (and I have no reason to believe it is), it needs a social solution, not a technical one.

On the other side of the fence:

3 hours ago, stgatilov said:

Git, on the other hand, has lots of unsafe ways to edit history, and it is often encouraged.

It does have unsafe ways to edit history, but they are not particularly "encouraged". Most Git guides I have seen make clear that rewriting history is very dangerous, completely screws up other people's work, and should be avoided whenever possible. Sites like GitHub even have the ability to administratively disable force-pushes altogether to stop people from doing them accidentally.

Quote
  1. Git is inherently harder to use than SVN. Moreover, it is harder not because it has to be so, but because it was ill-designed. I don't like the idea of forcing complexity onto everyone (including myself BTW).

Misconception: "Git adds a whole load of distributed functionality making it hard to use".

Reality: For the most common use case (pushing to a single central server), there is absolutely no need for any Git user to deal with  "distributed" functionality. You can produce a SVN-to-Git cheat sheet that maps commands almost one-to-one, and stick to those commands entirely. Some of the commands aren't particularly well-named (e.g. using "checkout" to switch to a branch, or "add" to add changes to a file rather than adding a new file), but they are not complex to use, and you can always set up aliases if you find the default names confusing. Conversely, many operations are significantly easier (and faster) in Git than in SVN. Only recently did SVN add the ability to view the diffs of each change in the history, or to show which changes were pulled in by a merge, and these features are still slow and cumbersome, whereas Git (and Hg) did all of this from the beginning.

  • Like 1
Link to comment
Share on other sites

3 hours ago, OrbWeaver said:

Misconception: "Git adds a whole load of distributed functionality making it hard to use".

Reality: For the most common use case (pushing to a single central server), there is absolutely no need for any Git user to deal with  "distributed" functionality. You can produce a SVN-to-Git cheat sheet that maps commands almost one-to-one, and stick to those commands entirely. Some of the commands aren't particularly well-named (e.g. using "checkout" to switch to a branch, or "add" to add changes to a file rather than adding a new file), but they are not complex to use, and you can always set up aliases if you find the default names confusing. Conversely, many operations are significantly easier (and faster) in Git than in SVN. Only recently did SVN add the ability to view the diffs of each change in the history, or to show which changes were pulled in by a merge, and these features are still slow and cumbersome, whereas Git (and Hg) did all of this from the beginning.

I did not say that distributed functionality is what makes git hard to use. I said that git in particular is such that it is hard to use. Not because it had to be so, but because it so happened.

You seem to bring another misconception here which I often hear: "You are using svn up and svn commit with SVN, so in git it would be the same but also there would be git pull and git push, nothing hard at all. And you don't need other commands or much knowledge for daily use". The reality is that after a week or month of daily use, people will have a choice of either learning the substantial part of how git works (which is much more complicated than SVN) along with some advanced commands, or suffering from their limited understanding of git.

  • Like 1
Link to comment
Share on other sites

7 hours ago, stgatilov said:

The reality is that after a week or month of daily use, people will have a choice of either learning the substantial part of how git works (which is much more complicated than SVN) along with some advanced commands, or suffering from their limited understanding of git.

I lot depends on the client software too.

I make assumptions on my user experience that branching and merging is much better in git, but I wonder sometimes if it's Sourcetree vs. Tortoise as much as git vs. svn.

Link to comment
Share on other sites

I'm a little surprised that some people here view Alberto's suggestion as an attack because although his style might seem a bit brash, he is obviously only trying to help and it's not as if what he's suggesting is something that is totally out of line. He's passionate and wants to help, what's wrong with that?

If newcomers and/or new suggestions are met like this then what will be the future of this community? Opinion doesn't need to be attacked, it can be viewed as an opportunity.

  • Like 1
Link to comment
Share on other sites

I don't mind lectures, but I prefer better teachers. If you're preaching, you should at least be able to explain why something is so awesome that you became an evangelist. And this is not the first time when he's preaching, but he doesn't understand the concept too well.

  • Like 1
Link to comment
Share on other sites

On 12/20/2019 at 8:25 AM, peter_spy said:

I don't mind lectures, but I prefer better teachers. If you're preaching, you should at least be able to explain why something is so awesome that you became an evangelist. And this is not the first time when he's preaching, but he doesn't understand the concept too well.

Not to belabor this but it appears that he's gone and he'll probably not return which I think is a shame. We might not see eye to eye on forum etiquette but I think we can all agree that any potential collaboration is probably now gone as well.

Also, I loved that he posted videos instead of just writing. I mean, how can you not just love that ?

Edited by Jetrell
Link to comment
Share on other sites

@Jetrell Nah, I'm not gone. It's simply I realized that doing as in the peter_spy's thread, replying to every single point to justify my view, is pointless.

Either people listen or not. And in this particular topic I didn't took it personally at all, it seems that most of the annoyance comes from previous discussions where I didn't participate. Or that the people was expecting me to procreate the conversation infinitely like with in peter_spy's thread, which basically I believe it was a mistake on my side.

In summary: I said nothing else because nothing relevant had to be said ??

  • Like 2
Link to comment
Share on other sites

I get all arguments. What is the cost/benefit of switching? Technically, very little benefit at this stage. So i don't see the point in changing just for git. 

Ignoring the git vs svn debate... I think there are some specific upsides to adopting github however, especially to take advantage of its free CMS features and bots. As well as the increased visibility / gentle onramp that it will bring to TDM for new developers. I.e.

[I assume everyone knows github, but I've listed these anyway for clarity]

Build tools:

  • CI test + build + release bots 
  • Bots to tag/triage new issues and close stale ones etc. 

CMS:

  • Issue tracking with built in conversations
  • Commit history / insights 
  • Simple forking / PR model 

If you're like me, you probably enjoy following project activity on github, reviewing commits, contributing to issue discussions and occasionally jumping into code with pull requests. 

I say let Alberto have a crack at setting everything up on github, including build tools and then do a show and tell for the community, so the community can decide whether we want to make the official switch. 

 

 

 

 

Link to comment
Share on other sites

I'm not sure what you mean by "CMS". If this is CMS aka wordpress, then there is website and forums. If you mean wiki, then here is the wiki. If you mean bugtracker, then here is one. Switching from one system to another is always a lossy process. Also, pretty much everything I listed above on TDM side is better and more well-known than builtin solutions which GitHub provides. MediaWiki is the widest-known wiki engine. Mantis is very popular too, although today's trend is using the awful Jira-monster. Forum is probably not so good (I have never heard about our Forum engine until now). As for website, is there anything except for GitHub pages on GitHub?

This is perhaps not important for all the hipster developers hanging around, talking about trends and coolness... but I regularly browse through years-old history. I do SVN blame and read commits from 2008 explaining why something was done. I dig issues from TDM 1.04 times. I search over the forums to find what some stale shader was doing originally and how to use it. Migration from svn to git can save most of the history, but migrating other things sounds like an even more stupid idea.

 

Quote

CI test + build

Well, we have CI/autobuild (thanks to Cabalistic).

Quote

Issue tracking with built in conversations

TDM bugtracker is very good, much more detailed than GitHub tracker yet not monstrous like Jira. And it is filled with lots of useful history :D

Quote

Commit history / insights

SVN blame and SVN log is enough for history. Insights... is it the metric like who made more commits and which months was more "productive" ? ?

 

Quote

If you're like me, you probably enjoy following project activity on github, reviewing commits, contributing to issue discussions and occasionally jumping into code with pull requests

That's the reason why I suggested creating an official mirror of source code repo on GitHub. And don't forget about "increased visibility", which is also very important.

 

Quote

I say let Alberto have a crack at setting everything up on github, including build tools and then do a show and tell for the community, so the community can decide whether we want to make the official switch.

It would be a waste of time. There is already one such repo by Nbohr1more, and it gets constant warnings about its size (BTW, I have a reduced git repo with third-party libs excluded from history). Moreover, I think it is updated only manually now.

In order to set up a mirror, collaboration with our admin is necessary to provide automatic updates. Just converting the repo and putting it on GitHub would be useless.

  • Like 2
Link to comment
Share on other sites

You don't need to use GitHub/GitLab's bug tracker, wiki or continuous integration tools.

My proposal was using those platforms only for code storage, for easing merge requests. While breaking the different programs into different projects under the same account, for speeding up cloning repositories.

Nothing else.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Recent Status Updates

    • The Black Arrow

      Any of you heard Age of Wonders 4's OST?
      https://www.youtube.com/watch?v=Q0TcoMGq4iA
      I love how after all these years, Michiel van den Bos still conserves his "Melodic" spirit.
      · 0 replies
    • nbohr1more

      Moddb article is up:  https://www.moddb.com/mods/the-dark-mod/news/the-dark-mod-212-is-here
      · 3 replies
    • Petike the Taffer

      I've been gone for a while, but now I'm back, have a new desktop and I want to get back to making missions and playing missions. And doing other contributions. Waiting for my reset password for the wiki, but I'll take a look at it soon. Hello, all.
      · 4 replies
    • snatcher

      TDM Modpack 4.0 for The Dark Mod 2.12 released!
      · 1 reply
    • nbohr1more

      Congrats to all the busy Dark Mod developers! TDM 2.12 is here!
      · 4 replies
×
×
  • Create New...