Sonia

From PokerAI

Sonia is the tutor bot hosted on PokerParadime. Sonia has beaten both PokerAcademy's Sparbot and Vexbot with more than 3.5 BB/100h.

This is the decription of the pokerbot, quoted from [1]:

Sonia: World-Class Simulated Intelligence

Sonia provides the ultimate poker challenge for beginners and professionals alike. Developed by two Trinity College, Cambridge graduates with expertise in Behavioural Modelling, Cognitive Science and Artificial Intelligence, Sonia marks a milestone in the development of true Generic Expectation-Driven Intelligence.

Although Sonia came to existence as a result of four years research into the development of world-class poker playing software, she is by no means limited to playing poker.

A whole new field of truly conversational, goal-oriented, intelligent software applications have been made possible due to Sonia's entirely unique GRACES:

• Goals - Sonia's grounding in psychological modelling allow for a wide variety of new and more esoteric goal definitions. No longer limited to purely self-centric material goals like maximising financial gain or success rate, Sonia's ability to assess and control human expectations, beliefs or morale give her an incredible opportunity to enrich, challenge and entertain those who interact with her.
• Reads - By modelling individuals' behaviour as a function of their motivational expectations, traits and observational state, Sonia achieves an exceptional ability to rapidly assess individuals from the very first time of interacting with them based on her past experiences. This understanding of human behaviour enables Sonia to get inside the head of an individual to quite literally read their mind.
• Awareness - Sonia has an explicitly hard-wired sense of self. This gives her an acute awareness of her place in the environment with respect to both how her actions affect the way others think and feel, and how she is likely to react to situations in the future. Explicitly hard-wiring a sense of self also prevents the development of numerous pathological conditions that otherwise occur.
• Control - Before considering the purely observable consequences of a possible set of actions, Sonia first considers the psychological impact of her actions on the individuals in her environment. This ability to manipulate the expectations, beliefs or morale of individuals enables her to perform a number of mind-games and deceptive techniques in order to better achieve her goals.
• Empathy - Sonia has a unique ability to put herself in another's shoes and think from their perspective. This is done by literally shifting her sense of self to that of another. These empathetic skills make her the ideal individual to call upon for help, advice or tuition. Sonia's ability to stay in sympathy with an individual during tuition allows her to perfectly pace advice with respect to their current ability and to specifically focus on improving problem areas. Thought process analysis also allows for the generation of explicit explanatory motivations for any advice given.
• Synthesis - Sonia's ability to intuitively read individuals and situations enables her to mimic them and synthesise their likely behaviour, even in scenarios she has never seen before. These mimicry and imitation skills enable her to completely simulate whole environments containing multiple simulated agents for either testing, validation, natural selection (with or without trait mutation and/or inheritance), gaming or self-developmental purposes.

Generic

Sonia starts with no prior knowledge of the environment in which she is about to be immersed, prior to her initial training:

1. Past-Events Training Session - During this session Sonia is presented with unbiased historical event data, for example poker hands. The data will almost certainly be highly sparse and incomplete with much of the information having been concealed. Crucially, Sonia's method of learning is perspective-oriented in that her interpretation of events is based upon her current world-view. The result is that as she gains knowledge and experience from the events this in turn makes her better able to understand and interpret future training events.
2. Self-Immersive Sparring Session - During this session Sonia simulates her own simulated sparring environment where she gets to interact with self-synthesised agents. This session's sole purpose is to develop her sense of self, which is maintained separately to her understanding of others. We found that by explicitly hard-wiring the ability to view and sense self (almost like a virtual 'mirror') we prevented the development of any pathological behaviour or bad habits that otherwise occurred.
3. Ongoing Learning and Development - Once the sparring session has been complete, one or more (or all) simulated agents of the sparring session are replaced with real individuals. Continual learning and development is provided by combining the methods of the two previous trainings sessions. The sole purpose of isolating her from the real world in the first two training sessions is merely for her protection against exploitation in a hostile, or potentially costly environment.

Expectation-Driven

At the very heart of Sonia is a quest and desire to understand and predict specific human behaviour in terms of motivational expectations, traits and observational state:

1. Motivational Expectations - Motivational Expectations are beliefs centred on the future that may potentially act as a motivator or driver to act in a particular way. For example, in most games or sports your chance of winning varies throughout the game; you may fall behind and then catch up several times. Your behaviour may vary greatly depending on your current assessment of the likelihood of your success. Sonia assesses your current beliefs and expectations regarding a number of possible motivators in order to understand how these affect your resulting behaviour.
2. Traits - Each of us, unknowingly or not, exhibit personal character traits that define us as an individual. Sonia assesses individuals' behaviour to look for how strongly each individual exhibits certain traits and in what circumstances these traits become evident. People who share similar traits will often exhibit similar patterns of behaviour. Sonia's heightened ability to pick up on personal character traits allows her to rapidly assess new individuals and accurately predict their behaviour within an exceptionally small number of observations or interactions. This is to the point where she can convincingly mimic an individual's behaviour in both seen and unseen contexts.
3. Observational State - An individual's state will be affected by which observables they are privy to. The act of observing these modifies their own observational state.

They may be either public or private:

• Publicly Observable - In a game of poker the publicly observable state of the game would include such things as the current pot size, who last bet, who's the dealer, how many chips each player has etc.
• Privately Observable - In a game of poker each player also has his or her own privately observable hole cards.
• Combined with motivational expectations and traits, these three make up an individual's current 'Instantaneous State Of Being'.

Intelligence

Sonia's thinking and decision-making process is made up of the following two sub-processes:

1. Assessment of 'Instantaneous State Of Being' - Having achieved an understanding of how an individual behaves in relation to their 'Instantaneous State Of Being' we now wish to perform the reverse task of assessing their 'Instantaneous State Of Being' after observing an individual's behaviour. For example in poker we normally want to update our opinion of the likelihood of what cards an opponent might hold after each of the player's actions given what we know about the way they play. Fortunately Bayes' theorem gives us the mechanism by which we can update our beliefs based on new evidence.
2. Consequential Reasoning - Sonia only ever acts after first considering the consequences of her actions in the light of any goals she may have. When first pondering the consequences of one's actions its very easy to end up thinking of other individuals as inanimate, non-intelligent things which simply and predictably react in obvious causal succession. We often forget that individuals react not just because of the current altered world state, but also because of how they currently 'feel'; and your action may have just altered the way they 'feel' significantly. When Sonia considers the possible consequences of her actions, she first considers how her actions will modify any expectations and beliefs of any individuals affected by her actions. She then considers the detailed consequences from that revised perspective in order to pick the action that best satisfies her goals.

Links

• PokerParadime.com