Why is there "syscall" instruction in the x86-64 ISA, if syscalls are OS related?

Question

I am a bit confused about the use of syscalls in OSs, assembly, and machine code.

From what I have understood, syscalls are the interface for users and applications to request services and resources from the OS, and their implementation depends on the type of OS used in the machine. Their implementation is made in machine code.

So, why is there "syscall" instruction in the x86-64 ISA, if those are only OS-relevant?

Is it true, and if so what happens in the unpractical scenario where there is no OS on the machine? Is the "syscall" machine instruction unusable?

I have looked it up, and from what I have found most ISAs "guess" the use of OS in the computer, and by default provide fast and easy way to invoke those through machine code that directs the execution to a place in the kernel code that includes the implementatoin of the syscalls.

Answer 1

Because some OS features require a special hardware support. Without that support such OSes cannot run. So, CPU designers have to either provide those features – or know, that most modern OSes will never run on their system. Some low power CPUs do take that route.

Most modern OSes use "sandboxing" of user programs – they put each program into its own "sandbox", isolate it from other programs and from the real hardware. This protects the stability - a crash of one program will be contained, and will not cascade into a crash of other programs, or the entire system. It allows to run programs with different permissions on the same computer. It even allows to safely run programs that are somewhat malicious.

Those sandboxing features are almost infeasible without CPU support. The OS can never safely let the CPU to execute the code, knowing well that it will gladly send a "format all" command to the drive, if only some code from the Internet would tell it so. The only option left is to emulate the code in software. Software emulation is very slow, and there is something silly in the notion that we need a x86_64 emulator to safely run programs on a x86_64 CPU, don't you think?

So the CPU designers provide the features, that OSes need:

• virtual memory, so OS can provide a fake memory map to the programs, and limit their access to real memory and memory-mapped IO devices. This adds the related instructions: "enable/disable virtual memory", "modify virtual map".
• user ("sandbox") and kernel ("system") modes, so the program cannot run some instructions that can break it free from the sandbox (for example – the one that would disable virtual memory). This warrants "enter sandbox" and "leave sandbox" instructions.

The syscall is a "leave sandbox" instruction. It is necessary to run modern "sandboxing" OSes. There are some other ways to do a "leave sandbox" action (x86_64 itself has an older int instruction), but syscall was designed to do it faster.