AI for board game diplomacy

Agents cooperate, negotiate, and sanctions broken promises.

The success of communication and cooperation was important in supporting society to move forward through history. The closing environment of board games functions as a sandbox for modeling and communicating interactions and communication. You can learn a lot from playing them. Today, a recent paper published in Nature Communications says that artificial agents use communication and focus on Alliance Building. It shows how to cooperate.

Although diplomacy has a simple rule, it is difficult because it has high urgent complexity due to strong mutual dependence between the player and its immeasurable action space. In order to solve this issue, we have designed a negotiating algorithm that enables agents to communicate with the joint plan and agree.

It is especially difficult to cooperate when we cannot rely on our friends to do what they promise. Use diplomacy as a sandbox to find out what will happen if the agent deviates from the past agreement. Our research shows the risks that complex agents can misunderstand others about their intentions or misunderstand future plans. What are the conditions for promoting reliable communication and teamwork?

The strategy that approves the colleagues that breaks the contract indicates that abandonment of commitment dramatically reduces the benefits gained, which promotes honest communication.

What is diplomacy and why is it important?

Games such as Chess, Poker, Go, and many video games were always fertile basis for AI research. DIPLOMACY is a seven player game for negotiations and an alliance, and is performed on an old European map divided into a state that controls multiple units (diplomatic rules). The standard version of the game called Press Diplomacy includes the negotiations in each turn, and then reveals the movements selected by all players at the same time.

The core of diplomacy is in the negotiations, and the player tries to agree to the next movement. For example, a unit can support another unit and to overcome other units as shown here.

The calculation approach to diplomacy has been investigated since the 1980s, and many have been investigated in a simpler version of a game called Plazes diplomacy, which is not allowed for strategic communication between players. Researchers also propose a computers -friendly negotiating protocol that is sometimes called a “restricted press.”

What did we study?

Diplomacy is used as an analog of actual negotiations and provides a way for AI agents to adjust the movement. We adopt non -communication diplomatic agents and provide a protocol for negotiating contracts for a joint action plan, increasing their diplomacy in communication. We are called the baseline negotiators of these expansion agents, and they are restrained by their agreement.

Consider two protocols. Details will be described in detail about the Proposal Protocol and the proposed Choes Protocol. Our agent applies algorithms that identify useful transactions by simulating how the game develops under a variety of contracts. The NASH negotiation solution from the game theory is used as a principle basis for identifying high -quality contracts. Since the game can be deployed in many ways according to the player’s action, the agent uses the Monte Cal Rosimulation to see what will happen in the next turn.

Our experiments indicate that our negotiating mechanisms can greatly exceed the baseline non -co -sympathizer.

Agent breaks the agreement

In diplomacy, the agreement made during negotiations is not binding (communication is a cheap story). But what happens if an agent who agrees for a contract deviates from the next turn? In many real settings, people agree to act in a specific way, but will not be able to meet the commitment later. To enable the cooperation between AI agents or between agents, you must find out how to improve this problem and how to improve this problem. We have studied how to use diplomacy to identify the conditions for the abandonment of commitment to eroded and cooperate and promote honest cooperation.

Therefore, we will consider a deviation agent that overcomes honest baseline negotiators by deviating from aggressive contracts. The simple deviation simply agreed to the contract and agreed to move as desired. The conditional deviation is more sophisticated and optimizes the action, assuming that other players who accept the contract will act accordingly.

It indicates that simple and conditional deviations have greatly exceeded the base line negotiator.

To be honest, I encourage the agent

Next, use a defensive agent that responds to the deviation to work on the deviation problem. We investigate binary negotiators simply blocking communication with agents that break the agreement with them. However, since Shaning is a mild reaction, it does not neglect betrayal, but instead changes the goal to actively reduce the value of the deviation. It indicates that both types of defenders reduce deviation, especially the benefits of licensed agents.

Finally, we will introduce some games that have adapted, optimize the actions to sanctions agents, and are trying to reduce the effects of the above defense. The deviation of the learned will break the contract only if the immediate profit from the deviation is high and the ability to retaliate by other agents is low. In fact, the deviation of the learned can break the contract in the second half of the game, which achieves a smaller advantage than the approved agent. Nevertheless, such sanctions encourage the learned devastates to respect 99.7 % or more of the contract.

In addition, we will examine the possibility of sanctions and deviations of learning dynamics. What happens if the licensed agent may deviate from the contract, and a potential incentive that stops sanctions when this behavior is expensive. Such problems may gradually eroded cooperation, so you may need to repeat the interaction in multiple games, or to use additional mechanisms, such as the use of trust and reputation systems.

Our paper has many questions for future research. Is it possible to design a sophisticated protocol and promote honest action? How can we handle communication techniques and incomplete information? Finally, which mechanism can stop the destruction of the agreement? The construction of a fair, transparent and reliable AI system is a very important topic and an important part of the DeepMind mission. Studying these questions in a diplomatic sandbox can help you better understand the tension between cooperation and competition that can exist in the real world. Ultimately, we believe that by taking on these issues, we will better understand how to develop AI systems according to social values ​​and priority.

Read our complete dissertation here.