Linux Kernel Virtual memory space, Address Handling and Virtual Memory Confusion

Question

I am trying to implement the Meltdown vulnerability on a linux intel x86 laptop for a class demo. In the machine I am using I have nokaslr and pti=off in the kernel parameters.

I have created a kernel module that creates a global static char array called secret_array_kernel and then printk the address of this array. In my terminal I then retrieve the address using dmesg. For example 0000000054b7a361

I then take this address and copy and paste it into a program called MeltdownAttack.c that tries to find the contents of each index of this string using Meltdown.

I run the program with the given address of the array in this case 0000000054b7a361 only to find out it does not work. I repeated running the program but was unable to get it to print out the correct value once.

My first step in debugging was to create a global static char array called test similar how I did in the Kernel module but now this array lives inside of MeltdownAttack.c. I passed the address of this new array and I was able to get the contents of this array using the same functions as I tried with address of secret_array_kernel.

From what I understand in terms of virtual memory the array test lives in the processes user virtual memory, while the array secret_array_kernel lives in the processes kernel virtual memory. I know some of the Kernel Virtual Memory is Identical for each process but some is different for each process..

My kernel module looks something this

//...
static char secret[8] = {'M','Y','S','E','C','R','E','T'};
//...
static __init int test_proc_init(void)
{
   // write message in kernel message buffer
   printk("secret data address:%p\n", &secret);      
   printk("secret data value  :%s\n", secret);      

   secret_buffer = (char*)vmalloc(8);

   // create data entry in /proc
   secret_entry = proc_create_data("secret_data", 
                  0444, NULL, &test_proc_fops, NULL);
   if (secret_entry) return 0;

   return -ENOMEM;
}
//...

To then get the address I do

$ sudo insmod MeltdownKernel.ko
$ dmesg | grep 'secret data address'

//prints
[  485.147336] secret data address:0000000054b7a361

I have the following questions

• I am unsure of the address given by dmesg is the address of the array in virtual memory or physical memory (I believe it to be virtual memory).
• If this address does live in virtual memory And lives in the kernel virtual memory. I am unsure if the address of the array secret_array_kernel that was printed out by dmesg lives in the part that is identical for each processor or lives in the part that is different for each process.See image here
• I am also unsure if its okay for me to use the exact same address that was given by the kernel module after I ran dmesg. To then be used inside MeltdownAttack.c. (Do I need to put some sort of offset?)

Clearing up this will help me deduce if I either try to fix my code again or just modify the address im using.

I am sorry if I have bad grammar in the post. English is not my first language.

Answer 1

I am unsure of the address given by dmesg is the address of the array in virtual memory or physical memory (I believe it to be virtual memory).

Since you executed printk("secret data address:%p\n", &secret), you get the virtual address of the variable secret which is a pointer (you get the address at which the pointer is stored so the address of the address of the first char of your string). You should use &secret[0] instead if you want the address of the first letter of your string. The value at that address should be 'M' in ascii numbers so 0x4D in hex.

If this address does live in virtual memory And lives in the kernel virtual memory. I am unsure if the address of the array secret_array_kernel that was printed out by dmesg lives in the part that is identical for each processor or lives in the part that is different for each process

The whole principle of pti=off is to turn off Page Table Isolation which is a preventative measure for Meltdown. Essentially, before PTI, the page table of each process had its upper half portion handled entirely by the kernel and always the same for each process. This allowed to have the kernel in the virtual address space (VAS) of each process and reduced overhead for caching management, process switching during system call and other things.

As I understand, since Meltdown was discovered, Linux kernel devs and the devs of other OSes implemented different page tables for the kernel to properly flush caches while executing in kernel mode pages since you now have to switch CR3 (base table register) inducing a cache flush. The CPU has to flush since all the virtual address space now becomes invalid because of the base pointer changing. This obviously introduced severe loss of performance but allowed to mostly mitigate Meltdown. This is called PTI.

Of course, since you turned PTI off and your code is executed from a loaded kernel module, the address returned is an address managed by the kernel but I don't understand why it is so low. Normally, I'd think the kernel's address space would comprise only the last GB of a 4GB virtual address space on a 32 bits x86 computer.

I am also unsure if its okay for me to use the exact same address that was given by the kernel module after I ran dmesg. To then be used inside MeltdownAttack.c. (Do I need to put some sort of offset?)

I didn't see your code so I can't tell what you are doing in Meltdown.c. I don't exactly understand your question but normally you'd need the address of your first string's letter. Of course I think Meltdown is a complex attack that involves caching and precise timing so, unless you know what you are doing quite well, you should probably reduce the granularity of your code to one byte and do it in assembly.

As shown in the Metldown paper, the attack consists of triggering an exception by having carefully crafted instructions after the instruction that caused the exception. That way, since the instructions are executed out-of-order, the instructions that come after (including the instruction that caused the exception) have most likely been executed already but their result should not be committed yet (for example they should not have modified registers or what the authors call the architectural state of the CPU)

What the authors propose is the following :

; rcx = kernel address, rbx = probe array
xor rax, rax
retry:
mov al, byte [rcx]
shl rax, 0xc
jz retry
mov rbx, qword [rbx + rax]

You have an array called probe_array that spans 256 pages one page for each value that a byte can take. They read an attacked address to RAX (in the kernel protected area) that they multiply by 4096 by shifting its bits left with shl. They then have what they call the transient sequence of instructions (the instructions executed even though an exception was raised due to out-of-order) write one byte in probe_array at an offset of RAX (the attacked address). Since RAX is a byte that can range from 0 to 255 and it is multiplied by 4096, one cache line is written which is one of the first byte of the 256 pages of probe_array. What is left to do is to have another thread running on any core (since you attack L3 which is shared) read every first byte of each page of probe_array and time its accesses. The lowest access time is a cache hit. The page number in probe_array where that cache hit occurs, is the value stored at the attacked address.

At least, that is my understanding from having read the Meltdown paper.