Oh what a flexible database

So apparently I’ve been doing way too much work with my flat file parsing of the Alsherok files. I forgot a very key component of my database of choice, PostgreSQL: ability to act as a non-relational database such as MongoDB while retaining the power of an RDBMS! I know, that last sentence is kinda geeky. Essentially, I just needed to grab the section I wanted, turn it into JSON, and shove it directly into the database. After that, I can call the data back out using a hybrid of traditional SQL such as:

Which brings back the fields

as expected.

I can even ensure uniqueness by adding a unique index on a particular branch of the JSON:

This provides a unique entry and fast searching because.. you know.. indexes and stuff.

Needless to say, I felt the need to refactor the other flat files I’ve finished and make them all spit out JSON instead of complex table structures 😉 Thankfully that went much faster than writing them originally. I may actually meet my end of the month deadline after all.

On flat files and parsing…

One of the biggest first steps of the Alsherok project, beyond basic high level architecture and categorization, is understanding the raw data from the world. Parsing the data into a format that can be easy manipulated, searched, and aggregated is key to several aspects of the project.

  1. We’re going to need models for mobs
  2. We’re going to need models for objects (weapons, armor, bags, potions, food, etc.)
  3. We’re going to need models for the general environment
  4. I like playing with data
  5. Understanding aggregates can help with prioritization
  6. Cataloged data can be transformed to meet any need

Unfortunately, all of this data is stored in individual flat files for each area. Sure, I could combine all of the files and make a huge list, but that would be unwieldy and harder to look through. Besides, if I just wanted data in one place to do basic searches, I could just use grep against the folder.

Instead, I decided to parse all of the files into a set of PostgreSQL database tables. After playing with doing it in C#, I decided the iteration time was too slow so I switched over to Python. I was able to get one set of data (the mobs) completed after a few evenings of poking it with a sharp stick, and I found that HOLY CRAP THAT IS A LOT OF MOBS! In fact, right at 2255! I won’t need that many models, but it certainly does make the task of weeding through them a bit more time consuming.

That being said, I did have some fun playing around with aggregating data on different points such as gender, race, class, and zone just to see what I was working with. Unfortunately, I’ve already blown away all the tables because I am working on parsing out the objects now, and the script is set up as an all-or-nothing deal. I may refactor it here soon to be more selective, but I am working on this with a priority on speed over re-usability.

Speaking of speed… I did have some issues with the length of time it was taking to parse, somewhere around 3 minutes for 111 files. I posted on r/Python for some feedback and got some excellent pointers on places I didn’t need to use regex, and an excellent referral to a Python profiler that instantly pointed out my bottleneck: the database connection. I was being slick and using a with statement so I wouldn’t have to worry about cleanup, but that required me to open the connection for every write. Once I moved that database connection to the top of the function and handled everything with a try:except block, I got the execution time of the script down to just over 3 seconds. Yay! But now I have to do objects, which will likely add a few more seconds.

Once my parsing code is complete, I will release it along with the DDL for the database tables so that anyone else playing around with the AFKMud codebase will have a jump start on ways to analyze their data or as a first step in switching the code to store and retrieve from a database rather than a flat file.

Moving forward with Alsherok Revisited

It looks like I will be moving forward with the Alsherok Revisited project. I have a couple of people interested in making it happen. so I might as well grab onto that while I can. Alsherok Revisited is now the primary focus of Squirrel Alienz.

My first order of business is parsing through flat files used to store all the MUD data. Each zone is its own file that contains all references to the mobs and items inside the zone… This is going to be interesting. I originally attempted to throw something together in C# and it was slowly started to work okay. However, my iteration time was too slow and there seemed to be a lot of overhead. So, I switched over to using Python. I am almost done with the first round of parsing which will net me the mobs (mobiles) and the mobprogs (mobile programs). I am storing everything in a PostgreSQL database because, you know, Postgres is pretty awesome (and open).

Once I have the mobs defined, I need to send them over for modeling review. Oi. There are so many models needed with this transition from text to 3D. Even the background descriptions will need to be represented. I can see a lot of vague translation going on here. Either way, better to get started earlier on this part. Plus having the mobs already stored in a database will allow me to pull some interesting statistics.

Integrating event dispatchers into the world tick

While it is perfectly acceptable to handle updates within character blueprints, sometimes it doesn’t make as much sense to perform all updates immediately when in a multiplayer experience. I previously wrote about creating a game tick that could be used for timed updates, but I didn’t have a hook into how to perform those updates. I created that hook by using an event dispatcher within the world tick logic. This allowed me to add events to the dispatcher queue and execute the queue at whatever interval the world tick was running.

So the logic works something like this…

Event happens → Add reference to an event to the event dispatcher → OnWorldTick, process everything in the dispatcher queue → Empty queue and start over

(more…)

Playing around with Unreal Motion Graphics (UMG)

I have been playing around with UMG since it came out in experimental format, but just got around to trying to tie data binding to a bar (for fun things like health, stamina, etc). UMG was promoted from experimental status with the latest UE4 release, so everything should be working fairly consistently. I spun my wheels for a little bit because apparently I forgot basic numeric properties >.>

To calculate the percentage for the bar, you have to take current <whatever> and divide it by max <whatever>. Basic math stuffs, yeah? The function for the percentage expects a float returned, because that is standard percentage stuff. Well apparently if you architect your variables as integers and ignore the main difference between integers and floats (whole numbers vs. fractions), you end up with a bar that only registers 0 and 100%! Whoops.

Go maths!

Besides that little snafu, I found the data binding within UMG to be extremely easy to handle inside of Blueprints once I figured out the casting I needed to do. I eventually switched to getting my character’s attributes through a Blueprint interface instead of having to mess around with so much casting. I found it much cleaner and easier to work with.

For example, I created an interface and implemented it in my character class like this:

implemented-interface

After which I can easily pull data out in the UMG HUD class like this:

called-interface

The result of the bars on the front end looks something like this:

bar-output

I’ve since created new attribute variables as floats and recreated the workflows (UE4 doesn’t allow you to change data types of variables that are in use anywhere in the Blueprint). I imagine as long as I segregate my interfaces well enough, they won’t become too unmanageable.

Game Tick – explained

I felt like my previous post deserved at least a little bit of a breakdown as to how to implement the game tick. If you have any desire to create an RPG-derivitive of some sort and your characters regenerate slowly over time rather than instantly, the game tick is your best friend. If you don’t like the idea of processing everything at the same time, you can always implement multiple ticks to handle different aspects of the update. Or use some type of tick logic on individual effects. There are many ways you can go with something like this, but deciding how you want to update your characters is very important.

I figured it would be worth it to run through the breakdown of how this tick is set up so that anyone that wanted to follow along would not need to just look at the screenshot and just blindly implement it hoping for the best.

Let’s start at the beginning…
UE4-BeginPlay

Event Begin Play issues a call to Game Tick, which is a custom event. One improvement to this might be adding a Switch Has Authority node so that you don’t make unnecessary calls. That node is on the Game Tick, so regular clients will get no further anyway.

UE4-GameTick

GameTick first makes sure that the entity trying to create a new tick is the authority (wouldn’t want peons defining their own ticks, right?). After that it seeds a new random stream. I have no interest in repeatable random, so the stream is set up to get a new random each time. I still need to fix the seed variable because it has no logic to regenerate every time, but the framework is there. It then gets a random integer between the min and max variables using the random seed and sets the CurrentGameTick value.

UE4-EventTick

Next, EventTick is obviously firing off all the freaking time, but we really don’t want to update that often. To fix that, we give it a sequence (so we can also process other stuff later) and check if the CurrentGameTick is not equal to 0 (ensuring that we have at least set it once). If it is something other than 0, we set a Delay for the length of time. After that delay, we execute another custom event, ProcessTick.

UE4-ProcessTick.

The last step here is to actually process events for the Game Tick. ProcessTick should contain whatever callouts to process things that you want to do every tick.

Finally, a quick look at the variables used here
UE4-GameTickVariables

Ignoring DefaultSceneRoot, the GameRandom is a VariableStream, while the rest are Integers.

There you have it. Hopefully an easy-to-follow implementation of basic game tick functionality in your game, using only Blueprints 🙂

As a recap:
1) Begin Play - Call the Game Tick
2) Game Tick - Create a new value in some range based on random seed
3) Event Tick - Wait for Game Tick time
4) Process Tick - Do stuff

Let me know if there is something here you would like me to explain more clearly 🙂

Game Tick

So I was messing around in Unreal Engine 4 and I managed to create the basis for my game tick through the Blueprint visual scripting system. For those of you unfamiliar with the concept, the tick (not confused with the constant tick for every frame in 3D rendering) is when the major game logic happens. The tick is when a player regenerates… when damage effects are applied… when regularly decremented values are assessed. Basically the tick is the master of all.

It actually wasn’t too bad in Blueprints. I am actually fairly proud that I managed to get the skeleton working the first time through without a bunch of trial and error. ^_^ Here is the Blueprint as it stands right now:
UE4-GameTick

This is all implemented in the Game Mode Blueprint that is supposed to contain the rules for the game. It should be fairly straightforward to implement. If you have any problems with it, let me know.

Unreal and the death of Unity

Among other things that I’ve been doing recently (changing hosting providers, playing with site design, working my way through titillating finance classes…), I’ve completely changed my game engine of choice. Make no mistake, the decision was not easy or flippant. Over the past couple of years I invested quite a bit of time learning the Unity engine and money to acquire extended capabilities (visual finite state machines, dialogue management, etc) and assets.

Bear with me a moment and I will explain my reasoning (or excuse). When I was first getting serious about game development I tried as many different engines as I could get my hands on. Some were free or low-cost, while others started in the tens of thousands to even start licensing for use. I tested Torque 2D/3D, Game Maker, Unigine, Unity, UDK3 and quite a few others.

I eventually settled on Unity because they had the polish that tools such as Torque lacked along with an active and supportive community. Their licensing was also fairly straightforward: Indie license for desktop platforms was free, Pro license and mobile platforms had a cost. Shortly after I started using Unity, they also released mobile platforms under the Indie license for free.

“That’s awesome!”, you say.
“Aye, I agree!”, I concede nonchalantly.

But that’s not the whole story. Certain functionality, such as calling a web service was restricted to pro license only. I found that out during a hackathon where my team created a game in Unity to delete spam mail. When I tried to build it for mobile, it failed because we had made a call-out to an external web service. >.< >.< >.< <---- those are all of my angry faces because I didn't realize it earlier.Along with some of the other limitations of not having pro, there were several plugins that would be quite useful, but absolutely wouldn't work without a pro license. Okay, so mark those off for now.Even with all of the limitations, I was still plugging along with Unity because they were still the best option available. Sure, I would probably need to shell out $4500 ($1500 for pro, $1500 for android, $1500 for iOS) if I ever wanted to release anything worth releasing. But that is a fixed cost, right? Sure, if you never want to get any major version upgrades. Those licenses cost another $600-$750 for each platform. Don't worry, the cost will all make more sense in just a moment.Cue a post a couple of months ago from Epic Games, creators of Unreal Engine... they are releasing a brand new engine, Unreal Engine 4, with a brand new licensing agreement. To be honest, If UDK3 hadn't had a Windows-only-and-25%-of-all-profits-over-$50,000 type of license, I might have considered it! Tentatively I continued reading the information... $20/month subscription (can cancel anytime and still use the engine), FULL SOURCE CODE ACCESS <-- can you tell I'm calling that out?, and 5% of gross sales paid back to Epic.Bomb. Dropped.To put it in perspective, I would need to sell somewhere around $90,000 with Unreal to equal the cost of the Unity pro license. By that time, I wouldn't care much what engine it was created in... I would just be enjoying a successful release.Considering my loss was only about $20 if I didn't like the product, I signed up the moment I had a chance. It didn't work at first on my Macbook, which is my normal development device. No worries. They promised support on Windows, Mac, and Linux. It worked perfectly fine on my Windows desktop. Within the next month they had released a huge boatload of updates, including one that made the editor work perfectly on my Macbook. The last release included hundreds of Epic + community updates. They are moving toward real-time updates for people with an active subscription (because they provide Github access).Needless to say, I haven't found a reason to stop the subscription yet. I have been devouring their videos (all professionally done), and working on porting my existing work over to their platform.Let's go back to the source code thing. That one is important. Everything is in C++ (low-level and fast). If I don't like something about the engine, I can get rid of it. I can completely replace any part of the engine because I have full access to the source code. Do you know how ridiculous that is? Several companies in the game industry charge fees starting in the hundreds of thousands to access that type of information.

Beyond the benefits I’ve already stated, I noticed another big thing. Everything I need to create a game is already right in front of me. This is an engine created for game developers, by game developers, with the intention of providing the entire workflow. This is a system they would use to actually create a game from start to finish. I don’t need to spend a bunch of money acquiring basic extensions to functionality to help with visual scripting or lines in the editor to help place things…. all of this stuff comes standard.

I don’t foresee Unity actually dying. They are a solid company and a lot of community support. But they died in my small universe. Going forward, I have focused all of my efforts on Unreal Engine 4 development. I will take what I can along with me (assets may be about it), but I will need to recreate many things. However, this is a necessary evil to which I am looking forward.