Cheat Engine Forum Index Cheat Engine
The Official Site of Cheat Engine
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 


RIP-Relative Addressing and Jmp Oddities (64-bit)

 
Post new topic   Reply to topic    Cheat Engine Forum Index -> Cheat Engine Tutorials -> Auto Assembler tutorials
View previous topic :: View next topic  
Author Message
ParkourPenguin
I post too much
Reputation: 138

Joined: 06 Jul 2014
Posts: 4275

PostPosted: Sat Jan 28, 2023 11:38 pm    Post subject: RIP-Relative Addressing and Jmp Oddities (64-bit) This post has 1 review(s) Reply with quote

Instructions can access other areas of memory by directly specifying an address:
Code:
mov [00473518],ecx
More often than not, a symbol is used instead:
Code:
mov [my_address],ecx
mov [game.exe+1378C],ecx
Cheat Engine automatically replaces the symbol with the address it is associated with.

Sometimes these instructions work fine. Other times an error such as "offset too big" occurs. This post goes into the details on why this happens and how to fix it. There's also information on a very similar problem regarding `jmp` instructions at the end.


Explanation

The instructions people read and write are a human-readable version of machine code- the bytes executed by computers.
Code:
 bytes    -   instruction
48 89 0F  -  mov [rdi],rcx
An assembler turns instructions into bytes. A disassembler turns bytes into instructions.

Not all instructions can be assembled. For example, this instruction is impossible:
Code:
mov [eax],[ebx]
While it might look fine to a human, the CPU can't do it.

"Impossible instructions" are impossible to assemble. There is no machine code that does what the instruction asks the CPU to do. The CPU has its limitations just like everything else.
The fact that directly accessing an address works only sometimes is one such limitation:
Code:
mov [00473518],ecx
Sometimes it's possible, other times it's impossible. The rules around this can seem a little complicated.

In 64-bit code, there are two ways it can be done:
  1. The address being accessed is "near" the instruction accessing it.
  2. The address being accessed is "near" the beginning of the address space.
The first one is refered to as "RIP-relative addressing".

RIP is a register called the "instruction pointer." It holds the address of the next instruction to execute.
Code:
game.exe+1234  -  8B 34 CA  -  mov esi,[rdx+rcx*8]
game.exe+1237  -  8B 46 3C  -  mov eax,[rsi+3C]
When a thread executes the instruction at "game.exe+1234", RIP will be the address of the next instruction: "game.exe+1237".

As its name implies, RIP-relative addressing will use RIP to figure out what address to access. It does this using an offset. For example:
Code:
game.exe+1000  -  8B 05 02000000  -  mov eax,[game.exe+1008]
game.exe+1006  -  C3              -  ret
game.exe+1007  -  90              -  nop
game.exe+1008  -  10 27 00 00     -  (4-byte int)10000
Here, the RIP-relative offset in the first instruction is 2 (bytes 02 00 00 00). That's because the address being accessed is at "game.exe+1008", and RIP (the address of the next instruction) is "game.exe+1006". 1008 - 1006 = 2.
Search for "little endian" to learn why the bytes for the offset value are stored in reverse order.

The very important limitation here is that the offset from RIP is a signed 4-byte value. Addresses are 8 bytes. This means RIP-relative addressing can only access addresses near RIP.
Example:
Code:
200000000  -  89 05 ????????  -  mov [address],eax
200000006  -  ...
This instruction can access any address from 0x180000006 to 0x280000005. It is impossible to access any other address: the instruction `mov [170000000],ecx` can't exist at the address 0x200000000. It's simply something the CPU cannot do. Cheat Engine will complain with an "offset too big" error.
These minimum and maximum addresses come from the minimum and maximum values of a signed 4-byte integer. RIP = 0x200000006. RIP - 2,147,483,648 = 0x180000006. RIP + 2,147,483,647 = 0x280000005.

This is what was meant by "the address being accessed is 'near' the instruction accessing it." "near" specifically meaning about 2 GiB give or take.

That's it for RIP-relative addressing.

The second way is far simpler:
Code:
mov [address],eax
If the address is in the first 4 GiB of the address space (addresses 0 to 0xFFFFFFFF), it just works. It always works, no matter where the instruction itself is located.
Code:
200000000  -  89 04 25 24514000  -  mov [00405124],eax
It doesn't matter that the instruction is more than 2 GiB away and RIP-relative addressing can't be used. It works anyway.

There are some exceptions to the above two rules: for example, regarding the `mov` instruction, if eax or rax is used as the value being moved, it just works.
Code:
200000000  -  A1 0000000004000000     -  mov eax,[400000000]
200000000  -  48 A3 0000000004000000  -  mov [400000000],rax
This only applies to the `mov` instruction. `add`, `sub`, etc. can't do this.


Most of the time, directly accessing an address isn't an issue. Most Auto Assembler scripts only access symbols defined in the script itself.
Code:
alloc(newmem,2048)
alloc(my_address,8)

newmem:
  mov [my_address],rdx
  ...
This will always work. Cheat Engine will automatically put "my_address" near "newmem" since they're in the same script.

This can start to fail if a script accesses a symbol defined elsewhere.
Code:
// script 1
alloc(my_address,8)
registersymbol(my_address)
Code:
// script 2
alloc(newmem,2048)

newmem:
  mov [my_address],rdx
This might not work. There's no guarantee "my_address" will be anywhere near "newmem". Maybe they are close enough to each other, or maybe "my_address" will be in the first 4 GiB of the address space. In either case, it would be pure dumb luck if this works at all.

There are workarounds and ways to guarantee it will work.


Solutions

Address a memory location indirectly:
Code:
mov rcx,my_address
mov [rcx],rdx
Remember to backup and restore the register used with push / pop if necessary.

Combine scripts together. Instead of two allocs in two separate scripts, combine the scripts together as shown previously.
Being more explicit with labels can also work:
Code:
alloc(newmem,4096)
label(my_address)

newmem:
  mov [my_address],rdx
  // ...

newmem+800:
my_address:
  dq 0
0x800 bytes is more than far enough away to not interfere with the code of most code injections.

Use the third parameter to alloc. This parameter specifies an address to allocate memory near.
Code:
alloc(my_address,8,0x80000000)  // allocate in first 4 GiB of the address space
registersymbol(my_address)
Here, "my_address" can be used everywhere since it's in the first 4 GiB of the address space.

Be aware that the third parameter to alloc is most often used to fix a similar problem with `jmp` instructions.


Jmp instructions

`jmp` instructions jump to some other address. When the address is specified explicitly:
Code:

200000000  -  E9 FB000000  -  jmp 200000100
200000005  -  ...
The `jmp` instruction uses an offset: a signed 4-byte value relative to RIP. This is very similar to RIP-relative addressing.

It also comes with the same problem as RIP-relative addressing: if the jump destination is more than 2 GiB away, the instruction can't be assembled. Well... almost.

The "good" news is that Cheat Engine created a `jmp` pseudoinstruction that works even if the destination is far away. The bad news is that the pseudoinstruction takes up 14 bytes instead of 5. This is really bad news in code injections.

Code injections rely on the size of the jmp instruction being consistent. Using step 2 of the CE tutorial as an example, consider a typical AOB Injection script:
Code:
[ENABLE]
aobscanmodule(INJECT,Tutorial-x86_64.exe,29 83 F8 07 00 00)
alloc(newmem,4096,INJECT)

label(code)
label(return)
registersymbol(INJECT)

newmem:
  // injected code
  mov [rbx+7F8],#1000
code:
  // original code
  //sub [rbx+000007F8],eax
  jmp return

INJECT:
  jmp newmem
  nop
return:

[DISABLE]

INJECT:
  db 29 83 F8 07 00 00

unregistersymbol(INJECT)
dealloc(newmem)

{
Tutorial-x86_64.exe+2B4AF: B9 05 00 00 00           - mov ecx,00000005
Tutorial-x86_64.exe+2B4B4: E8 57 47 FE FF           - call Tutorial-x86_64.exe+FC10
Tutorial-x86_64.exe+2B4B9: 83 C0 01                 - add eax,01
// ---------- INJECTING HERE ----------
Tutorial-x86_64.exe+2B4BC: 29 83 F8 07 00 00        - sub [rbx+000007F8],eax
// ---------- DONE INJECTING  ----------
Tutorial-x86_64.exe+2B4C2: 48 8D 4D F8              - lea rcx,[rbp-08]
Tutorial-x86_64.exe+2B4C6: E8 45 DA FD FF           - call Tutorial-x86_64.exe+8F10
Tutorial-x86_64.exe+2B4CB: 8B 8B F8 07 00 00        - mov ecx,[rbx+000007F8]
}

In particular, these two parts:
Code:
INJECT:
  jmp newmem
  nop
return:
Above is the `jmp` that overwrites the original code at the injection point.
Code:
...
Tutorial-x86_64.exe+2B4B4: E8 57 47 FE FF           - call Tutorial-x86_64.exe+FC10
Tutorial-x86_64.exe+2B4B9: 83 C0 01                 - add eax,01
// ---------- INJECTING HERE ----------
Tutorial-x86_64.exe+2B4BC: 29 83 F8 07 00 00        - sub [rbx+000007F8],eax
// ---------- DONE INJECTING  ----------
Tutorial-x86_64.exe+2B4C2: 48 8D 4D F8              - lea rcx,[rbp-08]
Tutorial-x86_64.exe+2B4C6: E8 45 DA FD FF           - call Tutorial-x86_64.exe+8F10
...
Above is a comment at the end showing the instructions at and around the injection point.

The original instruction, `sub [rbx+000007F8],eax`, takes up 6 bytes. The `jmp` instruction is expected to take up 5 bytes, and the single `nop` instruction accounts for the leftover byte. This way, the "return:" label is located at the next instruction to execute: at the address "Tutorial-x86_64.exe+2B4C2".

However, if the destination of the `jmp` instruction (i.e. newmem) is located far away, Cheat Engine will automatically assemble the 14-byte `jmp` pseudoinstruction. This will completely screw up the injection point beyond what was assumed: more instructions get overwritten and the "return:" label might be in the middle of another instruction. Either of these problems would likely cause the game to crash.

This is the purpose of the third parameter to `alloc`. This parameter specifies an address to allocate memory near:
Code:
aobscanmodule(INJECT,Tutorial-x86_64.exe,29 83 F8 07 00 00)
alloc(newmem,4096,INJECT)
Here, the symbol "INJECT" is the address of the injection point (i.e. "Tutorial-x86_64.exe+2B4BC"). Passing it as the third parameter to `alloc` will guarantee the allocated memory will be allocated near the injection point. This in turn guarantees Cheat Engine will always use the 5-byte `jmp` instruction.

In rare cases, `alloc` might fail if Cheat Engine can't find any free memory near the third parameter. In that case, there's nothing that can be done but to force Cheat Engine to use the 14-byte `jmp` pseudoinstruction. The consequences of moving from a 5-byte `jmp` to a 14-byte `jmp` will need to be dealt with manually. In particular: figure out what new instructions get overwritten, adjust the signature of the aobscan, add the instructions in the original code to the code injection, fix the number of `nop` bytes at the injection point to align to the next instruction, and change the DISABLE section to restore all of the original instructions.

This is a complete example script that forces a 14-byte jump using step 2 of the 64-bit CE tutorial:
Code:
[ENABLE]
aobscanmodule(INJECT,Tutorial-x86_64.exe,29 83 F8 07 00 00 48 8D 4D F8 E8 45 DA FD FF)
alloc(newmem,4096)  // no 3rd parameter

label(code)
label(return)

registersymbol(INJECT)

newmem:
  // injected code
  mov [rbx+7F8],#1000
code:
  // original code
  //sub [rbx+000007F8],eax
  lea rcx,[rbp-08]
  call Tutorial-x86_64.exe+8F10
  jmp return

INJECT:
  jmp far newmem  // `far` forces 14-byte jmp
  nop 1 // 3 original instructions take up 15 bytes. `nop` padding = 15 - 14 = 1
return:

[DISABLE]

INJECT:
  db 29 83 F8 07 00 00 48 8D 4D F8 E8 45 DA FD FF

unregistersymbol(INJECT)
dealloc(newmem)

{
Tutorial-x86_64.exe+2B4AF: B9 05 00 00 00           - mov ecx,00000005
Tutorial-x86_64.exe+2B4B4: E8 57 47 FE FF           - call Tutorial-x86_64.exe+FC10
Tutorial-x86_64.exe+2B4B9: 83 C0 01                 - add eax,01
// ---------- INJECTING HERE ----------
Tutorial-x86_64.exe+2B4BC: 29 83 F8 07 00 00        - sub [rbx+000007F8],eax
Tutorial-x86_64.exe+2B4C2: 48 8D 4D F8              - lea rcx,[rbp-08]
Tutorial-x86_64.exe+2B4C6: E8 45 DA FD FF           - call Tutorial-x86_64.exe+8F10
// ---------- DONE INJECTING  ----------
Tutorial-x86_64.exe+2B4CB: 8B 8B F8 07 00 00        - mov ecx,[rbx+000007F8]
Tutorial-x86_64.exe+2B4D1: 41 B9 FF 00 00 00        - mov r9d,000000FF
Tutorial-x86_64.exe+2B4D7: 4C 8D 85 F8 FE FF FF     - lea r8,[rbp-00000108]
}
Minor note: the `call` instruction is similar to the `jmp` instruction in that it uses a signed 4-byte offset from RIP. Cheat Engine has a `call` pseudoinstruction that works around that limitation as well, but it's 16 bytes instead of 14.

In 32-bit code, none of this matters. Addresses are 4 bytes, so everything can be accessed using a 4-byte offset. All this only applies to 64-bit code where addresses are 8 bytes.

I hope this helps someone. Have a nice day.

_________________
I don't know where I'm going, but I'll figure it out when I get there.


Last edited by ParkourPenguin on Mon Jan 30, 2023 12:08 am; edited 1 time in total
Back to top
View user's profile Send private message
++METHOS
I post too much
Reputation: 92

Joined: 29 Oct 2010
Posts: 4197

PostPosted: Sun Jan 29, 2023 4:08 am    Post subject: Reply with quote

Thank you for writing this up. I have personally never experienced this issue, but I am aware of others that have.

Assuming that I am understanding this correctly, wouldn't having a custom injection type allow circumvention of this problem? For example, an injection type that made use of multiple lines of injection in lieu of a single line?

I think, even if such a measure would not offer a viable solution for this specific problem, that being able to 'bulk' inject would be useful anyway, as I have experienced situations where multiple injection points had to be manually consolidated into one script due to the fact that they were so close to each other and would have otherwise caused the target to crash.

Thanks.
Back to top
View user's profile Send private message
ParkourPenguin
I post too much
Reputation: 138

Joined: 06 Jul 2014
Posts: 4275

PostPosted: Sun Jan 29, 2023 2:50 pm    Post subject: Reply with quote

Most of the time, RIP-relative addressing isn't a problem. Most people already have the address they're accessing near the code injection itself- e.g. multiple `alloc` calls or labels.
This becomes a problem when accessing memory defined outside the script using it. e.g. two `alloc` calls in two different scripts might not be anywhere near each other.

I think what you're describing doesn't fix the problem. Regardless of what your code injections do or how they're organized, you still need to jump from the game's code into memory you allocated. You always have to be aware of 5-byte vs 14-byte jumps.

You could make a template that always uses a 14-byte jump and ignore the `jmp` problem completely, but overwriting more code comes with more problems.
JIT-compiled instructions with hard coded addresses would be harder to avoid. e.g. original code `mov rdx,[1071483CE0]`, address changes on restart. You could use reassemble / readmem, but that's annoying and most people don't know how to use those correctly. Weirder cases would require {$lua} to extract the address from the instruction manually, as restrictions around RIP-relative addressing might become a problem due to accessing that address from a different location in memory.
Any branches (jmp / jcc) to an instruction after the first in the injection point will crash the game. e.g. in the last script, any `jmp Tutorial-x86_64.exe+2B4C2` instruction that happened to be executed would jump to garbage code when the script is active. Very difficult to debug: breakpoints won't trigger, the game would appear to crash randomly. Using "Memory Viewer -> Tools -> Dissect code" would help, but that's only if you already know this could be a problem in the first place.

_________________
I don't know where I'm going, but I'll figure it out when I get there.
Back to top
View user's profile Send private message
++METHOS
I post too much
Reputation: 92

Joined: 29 Oct 2010
Posts: 4197

PostPosted: Sun Jan 29, 2023 7:24 pm    Post subject: Reply with quote

Thanks. I guess I do not fully understand the problem. For example, why does such a limitation exist in the first place?

If, for example, the code that you allocate needed to be placed somewhere far away from your injection point, is the 2GB/4GB limitation a matter of avoiding execution delays or something else?

Thanks.
Back to top
View user's profile Send private message
ParkourPenguin
I post too much
Reputation: 138

Joined: 06 Jul 2014
Posts: 4275

PostPosted: Sun Jan 29, 2023 8:10 pm    Post subject: Reply with quote

It's impossible to directly access an address more than 2 GiB away from the instruction accessing it.

For example, if you tried to assemble the instruction `mov [0x300000000],eax` at the address 0x200000000, it will fail to assemble. 0x300000000 - 0x200000000 = 0x100000000 = 4 GiB. It's simply an instruction that can't exist at that address.

There's a lot of extraneous information in the first post. I'll try to edit it down later.

_________________
I don't know where I'm going, but I'll figure it out when I get there.
Back to top
View user's profile Send private message
++METHOS
I post too much
Reputation: 92

Joined: 29 Oct 2010
Posts: 4197

PostPosted: Sun Jan 29, 2023 8:44 pm    Post subject: Reply with quote

Thanks. I will check back later, then.

I guess I still do not understand why it is impossible. This makes me curious about what would happen if you call something outside of the main module and place your code there, and whether or not such limitations would still be present. I assume that this would not be necessary or ideal.

It is not so important, I guess. I am mostly just curious.

Thanks.
Back to top
View user's profile Send private message
ParkourPenguin
I post too much
Reputation: 138

Joined: 06 Jul 2014
Posts: 4275

PostPosted: Mon Jan 30, 2023 12:09 am    Post subject: Reply with quote

Edited it. Removed most of the technical jargon, spaced out some of the wall-of-text paragraphs, added more code examples, and removed much of the extraneous information. It should read easier.
_________________
I don't know where I'm going, but I'll figure it out when I get there.
Back to top
View user's profile Send private message
++METHOS
I post too much
Reputation: 92

Joined: 29 Oct 2010
Posts: 4197

PostPosted: Mon Jan 30, 2023 6:48 am    Post subject: Reply with quote

Thanks. That clarifies a lot.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    Cheat Engine Forum Index -> Cheat Engine Tutorials -> Auto Assembler tutorials All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group

CE Wiki   IRC (#CEF)   Twitter
Third party websites