Source control lessons learned

by alarial | Jan 24, 2016 | GameDev, Unreal Engine 4

An important part of any distributed development team is an effective source code management (SCM) solution. Typically I’m a Git kind of person. I like the branch management and the simplicity of use, along with the wide variety of clients available.The specific SCM solution chosen is less important than whether the solution meets the purpose it is meant to serve. I am a big advocate of using the right tool for the job and there are some choices that would not have made sense for our development workflow. Thus, I was open to other solutions for game development purposes. My goal was to set up an SCM that facilitated development structure between myself and remote members that also integrated well with Unreal Engine 4 (UE4). This narrowed my choices considerably because I wouldn’t touch one of the three options, SVN, with your worst enemy’s face.

Public providers

Git is an option, but we ran into some problems very early on with this because arguably the most popular free and private-by-default provider, Bitbucket, enforces a limit on the size of the repository. This makes game development collaboration a bit harder because uncompressed game resources get pretty large pretty quickly. We hit our limits with a couple providers very quickly as we were bringing in assets to test and checking in our changes. These problems didn’t rule out Git, but it did cause me to reevaluate the solutions.

Bitbucket server

I still wanted to use Git, so I went out and grabbed the latest copy of Bitbucket server and installed it locally. I then fought the access battle of my overprotective network (by design) and a server trying to accept incoming connections. I was able to get it working from within the same network, but anything that required entry from the public internet died quickly. I tweaked settings for a while before deciding to move on because it wasn’t a good long-term solution anyway. i needed something that was not dependent on compromising my own security by exposing my IP address rather than routing through the VPN.

Perforce

I noticed when I first starting using UE4 that Perforce support was built into the editor from the start. I remember trying to get something working back at one of the hackathons, but getting everything up and running just didn’t materialize fast enough to consider it a viable option for that 24-hour period. I liked the concept of using it because from everything I have read, this SCM has a much wider adoption across larger game development shops and many other software companies. One of the advantages I see with Perforce is that it doesn’t care about repository size or large files (Git doesn’t like large files so much). So I decided to give it another shot at setup and see what I could make happen.

Perforce Cloud

My first attempt with Perforce was to use the Helix Cloud service they offered for free. This was only up to 5 GB, but it was plenty large enough to test out and see if we wanted to transfer over to a full installation. Unfortunately it didn’t work as planned. I was not able to get UE4 happy with the connection string and the information I was able to dig up online was not very helpful. Considering I didn’t know what a good installation looked like or how to solve problems with a cloud service I couldn’t touch, I decided to just go for the full thing myself.

Perforce Full Server Install

Perforce offers a free license for up to 20 users. This is awesome for smaller teams like mine and makes it really easy to get large-scale functionality at the micro scale of my team. However, because this is an enterprise-grade solution, that means the installation should be fairly difficult and convoluted, right? Not so much! The problems I ran into had nothing to do with difficulty installing or even basic setup. Let’s run through the issues I had getting a full Perforce server installation up and running and connected with UE4. I will point out all my servers were Ubuntu 14.04 for these installs so your experience may differ if using a different backend.

I looked up installation directions directly on the Perforce website. The website gave directions for adding the APT repository key to the keychain and adding the source to the sources list (which were actually correct and worked perfectly!). I had to install 1 package, helix-p4d, which installed a few other dependencies and then told me very simply how to configure a new Perforce server. After choosing some options, the server was installed and ready to go. What? Yes. The actual server setup on Ubuntu only takes like 5 minutes. So now we get into the issues I faced after server install.

1. My first installation was on a Digital Ocean droplet I already had out there. Everything installed fine, but I couldn’t reach it from UE4 for any amount of coaxing or sugary drinks. I was able to set it up in the Perforce Visual client (P4V) and add files to the depot, but connecting via UE4 just wasn’t happening. I tried messing with it for a while and finally decided that maybe my problem wasn’t my settings, maybe it was my location.
2. I set up an EC2 micro instance on AWS. Exact same installation steps because exact same OS. I was also able to connect via P4V client, but UE4 was having some problems. I started off with errors about the Unicode format. UE4 was complaining that a Unicode client required a Unicode server. First, I wasn’t aware that UE4 editor presented itself in a Unicode format. Second, I couldn’t find any information online about how to change this on the editor side. So, I turned to the server side and ran the command that was meant to irreversibly convert the installation to Unicode mode. It made no difference to the UE4 client. No matter what I tried to change or tweak, the client error would tell me that a Unicode client required a Unicode server.

What Finally Worked?

I set up a new EC2 Ubuntu server and chose Unicode support during Perforce server installation. My P4V client loved it! My UE4 client loved it! There was lots of love all around! After getting all this working, I had another team member validate my setup by having him go through just the client-side installation and setup procedure. As of last night, he was able to perform the setup successfully on both his laptop and desktop.

Conclusion

This past week I’ve felt pretty useless as I was not able to write any code, but getting source control working and validated from external clients was a huge win. This means we have a much easier path as we move forward and we don’t need to rely on proximity to onboard new team members (all code was hosted on my NAS previously). I think what surprised me the most was the lack of information surrounding Perforce setup with UE4. I found several guides and videos, but everyone essentially assumed a Windows installation for your local server, not a remote Linux derivative. To me that makes the most sense for remote hosting, and maybe it is just that larger studios don’t need online guides or tutorials. For me, it was a week-long frustrating learning experience that paid off eventually. I can only hope that all of my frustrations end up the same way.

I know I didn’t go into great detail as to setup or fixing of different pieces of this SCM experience, but we’re already approaching the standard college essay word count and I wanted this entry to be focused more on the troubles than anything else. Leave a comment if you have specific questions about this experience or the eventual solution.