Djprobe -- direct jump-inserting probe

Wed Jul 13 2005 
Author : Masami HIRAMATSU <hiramatu@sdl.hitachi.co.jp>

INDEX
1. Djprobes Concept
2. Installation, Configuration and Build
3. Usage of Examples
4. APIs
5. Sample Code
6. Limitations
7. TODO

1. Djprobes Concept
===================
 Djprobe provides the non-locking and ultra-light probe function. It uses the 
relative `jmp` opcode instead of `int3` break opcode. It can reduce overheads
of probing by the interruption, single-stepping and locking.
 As is well known, it is not assured that both a `jmp` instruction and desti-
nation address are inserted atomically. To avoid this atomic operation prob-
lem, djprobe bypasses the section under 'construction' by using kprobes.

 Djprobe has 3 running phases and the preparing phase.

- Preparing phase:
 1. Make a instance of djprobe.
 2. Copy the template of assembler stub code into the instance's instruction 
   buffer.
 3. Copy (Back-up) the instructions that will be replaced by the `jmp` 
   instruction into the middle of instance's instruction buffer.
 4. Set returning jmp instruction on the tail of the instance's instruction
   buffer.

- The 1st phase is the inserting phase. In this phase, a probe is driven by 
 the kprobe. The kprobe's pre_handler checks safety of the cpu on which it 
 works. If it finished checking all cpus, it goes to the 2nd phase. If not,
 it changes execution point to the head of stub code that the probe has.

- The 2nd phase is the djprobe working phase. In this phase, the probe is 
 driven by the djprobe. The kernel execution path is changed to jump into the
 stub code directly.

- The 3rd phase is the removing phase. In this phase, the probe is driven by
 the kprobe again. The kprobe's pre_handler checks safety of the cpu on which
 it works. If it finished checking all cpus, it removes the kprobe from the
 code. If not, it resumes execution.


2. Installation, Configuration and Build 
========================================
 You can choose how to install djprobe as a kprobes enhancement patch or 
 as a module.

 2.1 Install from the kernel source and a patch
 ----------------------------------------------
 1) Djprobe is a patch for linux 2.6.12. Download linux-2.6.12.tar.bz2 from
     ftp://ftp.kernel.org/pub/linux/kernel/v2.6/
 
 2) And untar it.
 
  $ tar xjf linux-2.6.12.tar.bz2
 
 3) Patch djprobe-2.6.12.patch to the kernel source.
 
  $ cd linux-2.6.12
  $ patch -p1 < $(SOMEWHERE_DJPROBE)/patch/djprobe-2.6.12.patch

 4) Ensure CONFIG_KPROBES and CONFIG_DJPROBE are set to "y" while configuring 
the kernel using make menuconfig/xconfig/oldconfig. 

 5) Build and install the new kernel.
 
 6) reboot

 2.2 Install as a module
 -----------------------
 1) You should create an environment for building kernel modules.
 It depends on the distribution.
 
 2) Build djprobe kernel module.

  $ cd modules/
  $ make

 3) Load djprobe module.

  $ /sbin/insmod mod_djprobe.ko



3. Usage of Examples
====================
 This package includes sample source codes and a helper script in examples/. 
After installation, you can build these sample codes just as followings:

 $ cd examples/
 $ make

 After that, you can get noop_kprobe.ko, noop_jprobe.ko and noop_djprobe.ko.
These modules are using kprobe, jprobe and djprobe respectively. 
 To load these modules, you must specify the address of inserting point.
In the case of noop_djprobe.ko, you must specify the size of instructions
replaced by 'jmp' code. 
 'disym.sh' script will help you to get that address and size. For example, 
if you want to insert the probes at the head of sys_gettimeofday() function, 
you will run that script as following:

$ ./disym.sh sys_gettimeofday

/lib/modules/2.6.12/build/vmlinux:     file format elf32-i386

Disassembly of section .text:

c011b1d0 <sys_gettimeofday>:
c011b1d0:       83 ec 10                sub    $0x10,%esp
c011b1d3:       89 74 24 0c             mov    %esi,0xc(%esp)

Please be sure that the above-disassembled instructions are relocatable.
Parameter: addr=0xc011b1d0 size=7

 Then, you can get the address of sys_gettimeofday() and how many instructions
replaced by 'jmp' instruction. The disym.sh script outputs the parameter which
should be passed to the noop_djprobe module. So, you can load the 
noop_djprobe.ko as below:

 $ /sbin/insmod noop_djprobe.ko addr=0x0xc011b1d0 size=7

!!CAUTION!!
 The djprobe will replace those instructions with `jmp` instruction. Thus you
should make sure that the instructions can be relocated to another address.
 The instructions those operate immediate values, address or registers
(except eip) are OK. But the instructions those operate instruction pointer
are NOT relocatable.
 For example, "mov" and "sub" are relocatable, but "jmp" will NOT be
relocatable.

Also, in the case of other modules,
!!CAUTION!!
 Djprobe can not work with other probes safely at the same address. 
 Please REMOVE noop_djprobe before inserting the noop_kprobe or noop_jprobe
 modules.

 $ /sbin/insmod noop_kprobes.ko addr=0xc011b1d0
 $ /sbin/insmod noop_jprobes.ko addr=0xc011b1d0



4. APIs
=======
 4.1 Djprobe Handler
 -------------------
 typedef void (*djprobe_handler_t)(struct djprobe *djp, struct pt_regs *regs);

 Type of handler function. Difference from kprobe is that some member of regs
are not referable. General purpose registers (e*x, e*i and ebp), flag register
and stack pointer (esp) are referable. But you must not read segment registers
and instruction pointer (eip). You can get the insertion address from djp->addr
instead of regs->eip. And djprobe's probe can be called from kernel segment.

 4.2 Djprobe Structure
 -----------------------
struct djprobe {
	void * addr;
	int size;
	djprobe_handler_t handler;
	struct djprobe_instance * inst;
};

 Structure for management djprobe. Members are described below.
  addr    : The address in which djprobe will be inserted.
  size    : The size of instructions which will be replaced by jmp code in 
            byte.
  handler : The handler executed when probe point executed.
  inst    : the pointer to the instance of djprobe.

 You should set appropriate values to 'addr', 'size' and 'handler'. 'inst'
will be set by register_djprobe() and unregister_djprobe() functions.

 4.3 Djprobe Interfaces
 ----------------------
 int register_djprobe(struct djprobe * djp);
 void unregister_djprobe(struct djprobe * djp);

 Registering and unregistering functions of djprobe. register_djprobe()
registers the djprobe structure specified by djp, allocates an instance and 
attaches the specified djprobe to the instance. It will fail if the specified 
djprobe is invalid, and/or failes to allocate an instance, and/or another 
djprobe is already registered at same address.
 unregister_djprobe() unregisters the djprobe structure specified by djp and
prepare to release its instance.

RETURN VALUE:
 -EINVAL : djp or its values are invalid.
 -ENOMEM : failed to allocate an instance.
 -EBUSY  : another djprobe is already registered at same address.


5. Sample Code
==============

--register--
        djp.addr = inst_addr;
        djp.size = inst_size;
        djp.handler = probe_func;
        if (register_djprobe(&djp) != 0) {
		/* error */
	} else {
		/* success */
	}

--unregister--
        unregister_djprobe(&djp);

--probing--
        void probe_func(struct djprobe *djp, struct pt_regs *regs)
	{
		printk("Probing(0x%p)\n", djp->addr);
	}


6. Limitations
==============
 - Djprobe can use just on i386 architecture.
 - Djprobe overwrites the instructions at least 5 bytes. These instructions
  MUST be relocatable. For example, `push` `pop` `mov`(arguments are immediate/
  registers).
 - Currently, you must specify the size of instructions which will be replaced.


7. TODO
=======
 - Calculate the size of replaced instructions automatically in the kernel.
 - Port to other architectures.
 - Support multiple djprobes registering at an address.
 - Check whether the specified address is already used by other probes.


Changelog:
* Wed Jul 13 2005 Masami Hiramatsu <hiramatu@sdl.hitachi.co.jp>
- First release.
