vds10

summary: add custom microcode instruction optimization rule

description:

Installs a custom microcode instruction optimization rule, to transform:

call !DbgRaiseAssertionFailure <fast:>.0

into

call !DbgRaiseAssertionFailure <fast:”char *” “assertion text”>.0

To see this plugin in action please use arm64_brk.i64

level: intermediate

Classes

nt_assert_optimizer_t

my_plugin_t

Base class for all scripted plugins.

Functions

PLUGIN_ENTRY()

Module Contents

class vds10.nt_assert_optimizer_t

Bases: ida_hexrays.optinsn_t

func(blk, ins, optflags)

Optimize an instruction.

Parameters:
  • blk – current basic block. maybe nullptr, which means that the instruction must be optimized without context

  • ins – instruction to optimize; it is always a top-level instruction. the callback may not delete the instruction but may convert it into nop (see mblock_t::make_nop). to optimize sub-instructions, visit them using minsn_visitor_t. sub-instructions may not be converted into nop but can be converted to “mov x,x”. for example: add x,0,x => mov x,x this callback may change other instructions in the block, but should do this with care, e.g. to no break the propagation algorithm if called with OPTI_NO_LDXOPT.

  • optflags – combination of optimization flags bits

Returns:

number of changes made to the instruction. if after this call the instruction’s use/def lists have changed, you must mark the block level lists as dirty (see mark_lists_dirty)

handle_nt_assert(ins)
class vds10.my_plugin_t

Bases: ida_idaapi.plugin_t

Base class for all scripted plugins.

flags = 16
wanted_name = 'Optimize DbgRaiseAssertionFailure (IDAPython)'
wanted_hotkey = ''
comment = 'Sample plugin10 for Hex-Rays decompiler'
help = ''
init()
term()
run(arg)
vds10.PLUGIN_ENTRY()