Embodied manipulation

Designing a control scheme for interacting with the world in a immersive sim open world adventure game

· 22 min read
Embodied manipulation

This is part of the series about my design for a systemic story game.

I love the controls in the original 1996 Tomb Raider. It feels like I'm touching the world. Holding down the keys for holding objects or holding on to ledges. Using the lower key for the legs and the upper key for the arms. It’s intuitive, consistent and it felt right for me.

Contextual buttons

My main frustration for controls are the contextual actions that depend on proximity to an object. I have been vaulting over campfires in Horizon more often than not just because the quicksave button is the same as the dodgeroll button. I think that the generic “action” button is bad. The action can be radically different depending on what pixel you are looking at in the case of two different targets next to each other. Games with auto-climb often make me climb over tables when I’m just wanting to go to pick something up. It’s another case of immersion-breaking when I’m with a group of companions indoors and trying to talk to them but accidentally climb and jump all over the place.

Keeping control

Most games don’t allow you to cancel, change or modify actions. You should be able to change your mind in the middle of anything you do. Never take away the control from the right stick camera or left stick movement. For example, if you tell the avatar to sit down, and that starts a 1.5 second animation, you should be able to cancel the action at any point, resulting in what would be physically reasonable. If you issue the action to drink, resulting in you reaching in your bag, taking out a flask, opening it and moving it to your mouth, you should be able to cancel that at any time. More important still; If you are knocked down during a fight, you should be able to steer yourself while falling, for the possibility to get back up on your feet faster. You should be able to grab hold of things. You should be able to roll to lessen the damage. It may be hard, but it should be possible to minimize the setback. Never take the control away from the player. Keep the basic things easy while providing lots of things to master.

More inspirations

Death Stranding uses L2 and R2 as grip buttons for the left and right hand. You can throw stuff by swinging while holding something and let go mid swing. The monkey climbing in Astro’s Playroom is wonderful, where you can reach for and grab handholds with L2 and R2, even using the motion sensor for the reaching and also for swing jumps. Uncharted 4 also lets you steer your hands when selecting where to climb. The Last Guardian use circle as the grip button, allowing you to push/pull and move with objects. The playstation symbol buttons should not be used for things where you also want to keep using the right stick. Dying Light has the right idea by using R1 to jump and holding on to ledges. Off-stick buttons should only be used for things that in-world would interfere with moving or looking around. Half Life 2 was influential in not restricting the movement while people were talking with you, even though you actually didn't say anything. Cyberpunk 2077 gives you the power to select things to say while still letting you look around and move at the same time. The quicktime events of Detroit: Become Human hints at something that could resemble embodied actions, but fails miserably. You should never have to state what button to press, other than as a reminder.

The original meaning of the playstation symbols are triangle viewpoint, square document, circle yes and cross no. With time, now they usually are used for triangle Interact, square Action, circle Evade and cross Jump.

Immersive Sim Open World Adventure Game Control Scheme

This is my suggestion for allowing many different actions but in a way that feels natural. Keep the action buttons reachable while keeping the thumbs on the sticks. Slower actions can require long-press.

Playstation Xbox Action
L2 left trigger Left
R2 right trigger Right
L1 left bumper Force
R1 right bumper Up
cross A Down
circle B Collect
square X Equip
triangle Y Think
L3 click LS Sprint
R3 click RS Inspect

Your hands

The L2 and R2 trigger buttons represent your left and right hand. The trigger buttons are analog, meaning that it registers gradual impressions from zero to one. The Playstation 5 controller triggers are also adaptive, meaning that they can resist at different positions to different degrees at different points. The adaptive trigger will create a connection between your hands and the hands of your avatar in game. You can feel when you pick something up, the force when you try to pull, push or lift something heavy, the thug when you catch hold of a ledge after a jump or fall, or the resistance while pushing open a door or pulling a lever.

Gradually pushing down the trigger will first raise your arm, ready the item in it and finally squeeze or use the item. Use the right stick to select a target and press down a trigger to reach and pick up that item. Releasing the trigger will put down the item at the place you are looking at, or just in front of you if you are not looking at a place within reach. If an item is just out of your reach, you will reach for it. Use the left stick to move or try to extend your arm towards the item if you're stuck.

The target for your hands is the item you are looking at that's within reach. The default setting would be to first subtly highlight the current target. When a trigger press starts the pickup movement, the target will be more clearly highlighted, and you will be able to cancel the action by releasing the trigger before you actually move the item.

If the target is stuck, you will place your hand on it. Stuck items include things that can’t be picked up, because they are heavy, equipped or connected to something else. You can squeeze the item by pushing the trigger to the max. Squeezing an item while looking at it will enable you to move it with your right stick.

You can move items between your hands. If you are holding R2 with an item in that hand, and then selecting that item by looking at it with the right stick, you may place your left hand on it by holding L2. Releasing R2 will move the item to the left hand. The looking part is optional if you have no other target within reach, the other hand is empty, and the item is not equipped.


Hold square to open your inventory as a radial menu of stuff near at hand. That menu will also have options for accessing your bags and other stuff. Select the item and release. Selecting an item from your inventory while not holding up a hand will equip it directly in its place or to the right hand. Selecting an item while holding L2 or R2 will put the item in that hand. Anything already in that hand will be put in your inventory or released if not collectable.

While holding an item, you can press square to equip it. When the item is equipped, you will not drop it by releasing the hand. Equipping something that isn’t wearable will make you hold on to it with the selected hand. Equipping a two-handed item will make you hold on to it with your left hand, if you can carry it with just one hand. Equipping something you wear on your arm or hand will put it on the opposite hand. If you are holding a ring in your right hand and equipping it, you will put it on the left hand. You can select the exact placement by raising your other hand and pointing with your right stick. Equipping other wearable items will put them on your body. You should also be able to look at all parts of your body by looking down on yourself.


While not holding L2 or R2, pressing circle will make you collect the item you are looking at, and put it in your inventory. You may hold circle as a shortcut for unlocking and opening containers.

Looking at an item within reach and holding L2 or R2 will pick it up in that hand. If the item is stuck, you will place your hand on the item. Having your hand on an item and pressing circle will make you try to unequip it and take it in your hand. For example, you would take your shoes off by looking down at them, hold L2 and R2 to place your hands on them, and then press circle to untie them and place them in your hand.

If not having your hand on an item, press circle while holding L2 or R2 to put the items you are holding in that hand in your inventory. Those items are always quickly reachable from your inventory by holding square.


Let's say you see a bow and arrows on a table. Look at the bow and hold R2 to pick it up. While holding R2, you can move it to your left hand by holding L2 and releasing R2. Having an item in your hand means that you use it, if it's the kind of item you use by holding it in your hand. You may also equip it with square so that you will not drop it by releasing the trigger.

Now hold R2 for picking up the arrows. While holding the arrows, press circle to store them in your inventory, or hold L2 to raise your bow while storing all but one of the arrows. Then hold R2 to draw the bow with one of your arrows. Release R2 to shoot an arrow.

You can throw an unequipped item by holding it up, hold L1 to aim and release L1 to throw the item. If the item is equipped, you must also release the hand to actually let go of the item during the throw. You don’t have to do the manual release or even the holding up of the hand if the equipped item has a stash of throwable ammunition. For a stash of equipped rocks, you may just hold down and release L1 to do a throw, as long as you have a free hand.

Having a rifle equipped means that you will raise it with L2 and use it with R2, using the same principles as for everything else. Gradually pressing R2 will raise your arm to the trigger. The adaptive trigger will push back at the position of the trigger, where the squeeze will fire the gun.

Reload the guns by releasing R2, look at the left hand and press square, meaning that you hold the gun in the left hand and use the right hand to re-equip it. If no other target within reach is selected, you may just press square while holding up the gun. Though you still need an empty hand.

This will also allow having one item in each hand, be it single-handed guns, melee weapons, throwables, or any other type of item, like a torch, compass, rope, or a brass key.


Holding L2 or R2 will block with whatever you have in your left or right hand. Time pressing L2 or R2 to parry. All of these are variations of raising your left or right arm.

Melee weapons can be used by squeezing. Hold squeeze for a strong attack. Modify the type of attack by the direction you move with the left stick and the movement of your pointing with the right stick, for swinging to the left, right, up, down. If your opponent parried your attack, you may follow up with a riposte by pressing L1.

Use L1 to force your movement. L1 while moving towards a target within reach will kick or tackle the target. L1 in a direction with no target will do a short dodge in that direction. L1 while standing still will apply force towards the target without moving. Do L1 without a target for a nice grunt.


Press cross to crouch. Hold cross to go prone. Press cross near cover to hug cover. Then cross again to crouch if standing behind cover. Press cross while crouching to duck. Hold cross while prone to hold your breath and play dead.

While prone, you can press R1 to go back to a crouching position or hold R1 to stand up. While crouching and standing still you can press or hold R1 to raise up.

Pressing against an incline while crouching or prone and holding R1 will make you try to climb while crouching or prone.

Moving crouching or prone will be silent while not in sprint mode, meaning that your avatar will try to not do something that would make noises.


Pressing R1 will jump. Combine with L1 for different jump-attacks. Hold R1 while standing still to charge a standing jump. R1 + left forward in any direction will jump in that direction. R1 + left sideways or backwards will dodge and duck in that direction.

Press R1 + left while crouching to roll, or R1 + cross + left to roll while running. This will work in any combination. Begin jumping in a direction and press cross before your feet left the ground to modify a basic jump ro a roll, or press cross while in air to roll after you land. Press cross while in air to pull your legs up for clearing obstacles.

Combining moves with cross will take you away from the right stick, but I think that is ok since it's something that should be a bit harder and would limit your awareness of your surrounding, since you would focus on the ground.


Hold R1 while running at obstacles for parkouring over them. This will work for everything where you don’t have to hold on with your hands. Bracing or pushing off with your hands can be done even while holding items, as long as they aren’t fragile or heavy. You will parkour the thing you look at as long as you keep up your momentum. This will also work for things like ladders that are especially easy to climb.

For climbing with your hands, make sure to have them free. Look at something you can hold on to and hold L2 or R2 for grepping with the left or right hand. With your free hand, look at the next handhold. Move your body and arm with left and grab it. Release the other hand and repeat for the next handhold. You can use square to lock your latest grip if you want to hang on without holding down the trigger.

Your feet will follow along using the nearest footholds while climbing with your hands. To climb over an edge, hold on to the edge with both hands, move the left stick upward to press your body up and then hold R1 to climb up with your legs.

Press L1 for using your arms to thrust your body to something out of reach. While reaching for a handhold, press R1 to jump in that direction. Jumping while not reaching for or holding on to the environment will let you jump backwards.

You can climb down over an edge by looking at the edge and using an empty hand. Holding the ledge, you can continue to move forward to drop down and turn around.


You will pause at edges, but can continue for jumping down if it's a comfortable drop. For larger drops, you will have to initiate climbing or switch to sprint mode. This will also help you with balancing on stocks. You will not step of the ledge, but can still lose your balance.

You can center your mass to regain balance by raising your left or right arm. The degree of press on the trigger correspond to how high your arms are raised. Looking down at the stock or ledge and pressing a trigger will take hold on the ledge. You may also crouch or go prone to keep your balance.


Use L2 + R2 for putting your hands on the target item in front of you and move forward to push. If you have a grip, you can also drag backward or rotate to the side. Hold L1 to try to lift the object from the ground, or L1 + left to force the item in that direction.


Hold square to open your inventory as a radial menu of stuff near at hand. Select your rope with the left stick and release square to place the rope in your left hand. Ready the rope by holding L2. For binding to something, look at the item within reach and hold R2 and press square. Cancel the binding by releasing R2 and/or L2 before finished. Untie the rope by pressing circle.

Pull the item by moving backwards while holding R2. Moving while not holding R2 will just let the rope out til the limit of its reach. Hold L1 + left to pull the rope while standing still.

Hold R1 + left for climbing up/down the rope. While hanging on the rope with R2, use movement direction to run on walls. Press R1 to jump against the wall with your feets. Release R2 to let go of the rope, for a swinging jump.

Look at the rope and press square to lock yourself to the rope, in case you want to use your hands for something else. Press circle to free yourself again.

For throwing a lasso or grappling hook or similar, you can hold a rope with your left hand and throw the part in your right hand by holding and releasing L1. Press circle to collect the rope back to your hand and circle again to put it back in your inventory.


While looking at a stuck item in reach, you can hold a trigger to raise your arm and place your hand at the item. While your hand is placed, use right to move it left, right, up and down. left will still control your walking, giving you the option to move forward and backward or to the sides.

Squeeze the trigger to grab hold on the item. Use your sticks to move the item in any direction. Combine with L1 to jank the item with force, or use L1 by itself to just punch your target.

Push buttons by squeezing them. Levers can be pulled by squeezing them and moving with the sticks. Wheels can be rotated by rotating the right stick. You may try to loosen it by holding L1 and loot it with circle.

The shortcut for opening a container is holding down circle. Not using shortcuts, you will open a chest by placing your hand on the lid and move it with the sticks. Unlocking a chest would be done by equipping the key to your right hand, placing your left hand on the lock and then raising your right hand. Insert the key by squeezing R2 and then rotate it with right.

You can open doors in a similar way, or do it quickly by pressing L1 while you are pressing up against the door.


Press R3 to inspect your target. It will mark up situational relevant things in your environment representing the awareness and knowledge of your avatar, lingering for a while like short-term memory.

The thing you are looking at with right is your target. Inspecting will display more in-depth information of the current target, including tips for available actions and how the thing relates to your quests and places you know.

Inspecting your hand will let you rotate it with left, including the item you are holding. Holding the item in your left hand allows you to manipulate it with your right hand.


Hold square to get your tool from the menu. Hold the thing you want to modify in your left hand and process it with your right hand by holding R2, and directing your action with your right stick. You may for example put a knife in your right hand and take hold of a tree branch with your left hand, for making an arrow or spear.

Place things together by releasing an item you're holding while looking at the other item. Attach one item to another by having the thing you want to attach equipped to the right hand, place the left hand at the target or hold the target with your left hand, and then look at the target while you raise your right hand and press square.


The triangle is used for mind-stuff, like dialogue, quest-tracking or magic. Hold triangle to select what to do from a radial menu for fast action or go one step further for a more comprehensive list of options.

Careful or Fast

Press L3 to toggle sprint mode.

While in normal mode, you will behave civilized among people, actively avoiding bumping into things or people and avoiding climbing on stuff just by walking towards things. You will also refrain yourself from stepping over cliff edges and falling to your death.

Tilting left in normal mode will increase the movement to running after about two seconds. Bumping up against obstacles while not running will make you stop for a second and then find a reasonable way to continue to the place you’re looking at, by moving in a direction similar to the direction of left. You will go around obstacles if you can avoid them by one or two steps in either direction. If there's no way to step over or around the obstacle, you will walk over it.

By entering sprint mode, you will have to take responsibility for your own safety. You may still choose your actual speed by how much you move left, but the maximal speed will be much greater and you will also change your speed faster. You will automatically step over or press through anything that comes in your way. The main difference is your speed and lack of safety net and that you don’t care if you break things in your way by pushing them to the side.

You will still have to manually jump or climb over things that reach over your waist. You must still use L2 + R2 to grab hold of edges if needed during a jump. Engaging in combat in sprint mode will trade speed for increased vulnerability and faster stamina drain.


Written by Jonas Liljegren
Building modern web components on reactive state semantic graphs. Passionate about exploring unconventional methods in technology development to shape a better future.