As a software company specializing in providing Intelligent Artificial Players (IAP) for the video game industry; creating our own video game showcasing such IAPs had two very important and distinct functions:
Firstly, it served as a tangible proof of concept of our product at work, allowing video game companies to see the IAPs in action and to play against them, experiencing the different styles and techniques developed by the AI through it’s training.
The second aspect was to take us first hand through the laborious process of creating and publishing one’s own game from scratch. This gave us the invaluable experience of walking a mile (and then some) in our customer’s shoes. Understanding the pain points, limitations, setbacks, and how the industry is serviced was key for us to gain insight into our own customer-centric value proposition.
We purposely designed a game with five arenas in which the level of difficulty increases with the transition through these arenas, from the easiest to the hardest. Once can only advance to the following arena with certain amount of points tracked in the centralized, global game’s leaderboard.
The more experience a player gains, the more points they accumulate, and more prepared to battle in the next arena. A simple concept. However, and also purposely, none of the variables of the game (e.g.: game rules, time limit per play, values of point, etc.) change with each arena, they remain the same throughout, so the only real difference in the level of difficulty is the increased proficiency that the other players (the adversaries) provide. Achieving this with AIs is what we do!
Think about it in terms of traditional sports. The dimensions of the environment don’t change from match to match, nor do the rules of the game. Obstacles are not added as you progress in a tournament, and the abilities of adversaries are not artificially augmented either. Video games had to develop with all of the above because it is very hard to always have someone else with comparable abilities and expertise to play against in real time. The recent trend to multiplayer is such because video games continue to seek this competitor-driven challenge and entertainment that other aspects of increased difficulty cannot provide.
If we were to create this same game (the proof of concept mentioned above) without the ability to pour in hundreds of players ready to challenge other players at different levels and timezones with human-like play and varied techniques, it would have been a true and expensive nightmare to get any kind of traction. We would have given up and moved to something else, as I’m certain many smaller independent game developers sometimes do.
What we offer, and this is now backed by solid experience having gone through it ourselves, is exactly that ability: To pour into a game, new or old, from the start or later on; countless players (intelligent and artificial) ready to challenge other players (intelligent and human) at different levels and timezones, with human-like techniques that vary in style and proficiency from one to the other. So one way to consider our product for the video game industry is as a service, something like ‘levels of difficulty as a service’.
If the above is not the wishlist of any multiplayer game, I do not know what is... Well, I do know, add ‘paying repeat customers’ to that.
True, our IAPs are not paying repeat customers, however the many humans that have a personalized and slightly new experience every time they play against our IAPs have an increased potential to be.
At this point, and before we move onto the ‘how’, it is key to understand that an essential aspect of a successful IAP is that it is not the best player. Unlike Google’s Deep Mind (and other amazing companies of the sort), our primary objective isn’t to advance the field of AI. We are not using games as an environment in which we can test the evolution of AI in our search for the perfect problem solver. We need “imperfection”, otherwise unbeatable IAPs would not only be annoying for humans to play against, but worse: Boring! Sure, the few master players of the world would find an unbeatable opponent fascinating, but the rest of us (99.9% of the population) prefer to have fun and win sometimes.
Now, how do we achieve this IAP diversity? By now you probably know that we use Machine Learning techniques as well as Deep Reinforcement Learning and others to create and train our IAPs. Without getting too technical, we explored three different approaches in order to generate variety in IAPs playing a game: Interrupted Development vs. Goal Alteration vs. Next Best.
1) Interrupted Development, or how we like to call it, ‘half-cooked’. This is when we interrupt the training of a neural network and pick different IAPs from distinct segments at that point. Not completely untrained nor fully trained either, somewhere in the middle. This achieves different styles and levels, but sometimes there are displays of odd gameplay, not quite random but not very human-like either. Depending on the game, this approach can render very interesting IAPs or completely useless IAPs.
2) Goal Alteration, or training the IAPs with different goals. This is when we slightly alter the goal the AI is training towards in order to achieve the variety in styles and proficiency. For example, instead of training towards winning (being 1st), we let the AI train towards being 2nd, or not being last.
3) Next Best, or subsequent move preference. This is when we have a fully trained IAP and we instruct it to, for example, choose its second best move when playing at a certain level. This allows for a lot of flexibility and very interesting gameplay in certain games that need a clear and gradual level demarcation, since we can tailor groups of IAPs to choose the third best move, or fourth, for example, which naturally provides different levels of proficiency, which in turn positions them appropriately in the leaderboard.
All three are interesting in their own ways and a lot depends on the particular game we are working on. We are also exploring other techniques (a bit fancier and borderline esoteric) with our AI specialists and trying them out right on the games. The point is that we are all learning in this evolving field of AI and paired up with the fascinating video game industry, we are here to disrupt how AI is produced, provided, and serviced in digital environments. Levels of Difficulty as a Service through our Intelligent Artificial Players is just a great start, and we are loving it!