SCRYDOSBox-XReferenceInputsend_mouse

DOSBox-XInput

send_mouse

Move the DOS program's mouse pointer, press its buttons or drag, and read back where the pointer ended.

send_mouse(x=null, y=null, button="", action="", units="screen", expect="", seconds=null, target="", to_x=null, to_y=null)

Try it

On Bricks, the game this repository builds, running with the mouse in control of the paddle, after screen_map_find recognised the play screen. court is an area of the demo's screen map.
Ask

Put the mouse pointer on the middle of the court and show me where it ended.

Call
send_mouse(target="court", expect="MOUSE$", seconds=10)
Reply 5 of 13 fields
{
  "status": "ok",
  "pointer": {
    "screen": {"x": 240, "y": 100, "width": 320, "height": 200},
    "mouse": {"x": 480, "y": 100, "max_x": 639, "max_y": 199},
    "buttons": [],
    "bios_mode": "0x13"
  },
  "driver_arrived": true,
  "motion": true,
  "target": "court"
}

The pointer went to the centre of court: pixel 240,100 of a 320x200 screen, 480,100 in the mouse driver's own coordinates. driver_arrived speaks for the driver's pointer only; motion says the move also went in as mouse motion.

The whole reply
JSON
{
  "summary": "send_mouse: pointer at 240,100, program's own pointer not read; no button pressed; reaction seen",
  "status": "ok",
  "x": 240,
  "y": 100,
  "units": "screen",
  "route": "guest",
  "pointer": {
    "screen": {"x": 240, "y": 100, "width": 320, "height": 200},
    "mouse": {"x": 480, "y": 100, "max_x": 639, "max_y": 199},
    "buttons": [],
    "bios_mode": "0x13"
  },
  "driver_arrived": true,
  "motion": true,
  "verify": {
    "checked": true,
    "seen": true,
    "expected": "MOUSE$",
    "verdict": "TARGET",
    "line": "Bochs port E9h: MOUSE",
    "seen_before_waiting": "<volatile>",
    "lines_before": "<volatile>",
    "waited_s": "<volatile>",
    "lines_total": "<volatile>"
  },
  "target": "court",
  "screen": "play",
  "identified_at_stop": 2
}

Arguments

x integer (>= 0, <= 32767), optional
Where the pointer goes, across, in units. Give it with y; left out, the pointer stays where it is.
y integer (>= 0, <= 32767), optional
Where the pointer goes, down, in units. Give it with x.
button one of "", "left", "right", "middle", default ""
The button that acts: left, right or middle. Left out, the left one when an action is named, else none.
action one of "", "click", "double", "triple", "down", "up", "drag", default ""
What the button does: click, double, triple, down, up or drag. Left out, a click when a button is named, else no button.
units one of "screen", "mouse", default "screen"
screen, the default, counts character cells in a text mode and pixels in a graphics one, as screen_view does. mouse is the mouse driver's own coordinates.
expect string, default ""
A regular expression for the line the program logs when it reacts, as in send_keys. Left out, nothing is checked.
seconds number (> 0, <= 600.0), optional
How long to wait for the expect line, up to 600. Left out, 10; refused without expect.
target string, default ""
A control, a named area of the screen the latest screen_map_find recognised, in place of x and y. The pointer goes to its centre.
to_x integer (>= 0, <= 32767), optional
Where a drag ends, across, in units. Only with action="drag", and with to_y.
to_y integer (>= 0, <= 32767), optional
Where a drag ends, down, in units. Only with action="drag", and with to_x.

What comes back

pointer
Where the pointer ended: screen in screen units, mouse in the driver's coordinates with its range, the buttons held and the bios_mode.
arrived
Whether the pointer is on the point asked for. It comes only when something could check that: the program's own pointer, or units="mouse".
driver_arrived
In place of arrived for a plain screen move: whether the mouse driver's pointer got there. A program with a pointer of its own may not have followed.
asked
When the pointer did not get there, the driver coordinates that were sent.
motion
true when the move also went in as mouse motion, which a program that reads motion needs. Otherwise it says why not.
verify
Whether the program reacted, as in send_keys.
All 27 reply fields
  • cause (string)
  • regpoint_hit (object)
  • cpu_mode (string or object)
  • e9 (object)
  • frame (integer or object)
  • instruction (DosboxInstruction)
  • pc (string or object)
  • polls (integer)
  • regs (object)
  • stop_seq (integer)
  • watches (array)
  • action (string)
  • arrived (boolean)
  • asked (object)
  • button (string)
  • delivered (boolean)
  • driver_arrived (boolean)
  • identified_at_stop (integer or object)
  • motion (boolean or object)
  • pointer (object)
  • route (string)
  • screen (string)
  • target (string)
  • units (string)
  • verify (object)
  • x (integer)
  • y (integer)

What it refuses

Why it works this way

DOS programs read the mouse in different ways: some ask the driver where the pointer is, many read how far it moved, and some drive a PS/2 mouse themselves. So a move goes in twice: first as motion, which reaches all of them, then as an exact position. motion says whether the first half went in.

The pointer that moves is the mouse driver's, the one DOSBox-X provides: a mouse driver loaded inside DOS reads its own hardware and is not reached. A program that keeps a pointer of its own, adding up motion, can end somewhere else, so a plain screen move answers only driver_arrived.

A screen map is a file that names a program's screens and the areas of each. When it also says where the program keeps its pointer, the server reads that pointer and corrects the move, and arrived then speaks for it.

More on why

How programs read the mouse

DOSBox-X passes on your own mouse's motion only while its window has captured the mouse, which no session does. So the helper sends the motion itself, through DOSBox-X's integration device, then sets the position.

  • Motion counters (INT 33h AX=0Bh) and movement events (a handler set with AX=0Ch): reached by the motion.
  • A PS/2 mouse the program drives through the BIOS: reached too, as DOSBox-X scales the motion, so a move of a few units can reach it as none.
  • The driver's position: set exactly (AX=04h) after the motion.
  • A build whose integration device takes no motion only sets the position, and motion says why.

pointer in full

  • screen: x, y, width, height in screen units; unavailable in a mode of unknown size.
  • mouse: x, y in the driver's coordinates, with its range max_x, max_y.
  • buttons held, bios_mode, and vbe_mode under a VBE mode.
  • program: the program's own pointer when a screen map gives it: x, y, and corrections, how many times the move was corrected.
  • program.reach: when it did not get there, the span (x, y, low and high) it covered between the driver's two corners.
  • program.range_widened: the driver range (x, y) the server set, in the reply that widened it.

Under a VBE mode the BIOS mode byte is the emulated card's own number, so the helper asks VBE for the mode and its resolution (INT 10h 4F03h and 4F01h). The units are then that resolution: pixels, or character cells for a VBE text mode.

Each screen unit covers several driver coordinates, and the move aims at the middle of them. The driver rounds to its own steps and a program divides its range back, as Deluxe Paint does with eight coordinates a pixel. Aimed at a unit's first coordinate, the pointer could land in the unit before.

arrived is checked in the units asked: any driver coordinate inside the unit counts. It is false when the driver's rounding or the program's own range left the pointer in another unit, and asked then gives the driver coordinates sent. A text mode snaps the pointer to its character cell, and the reply reports where it ended.

A program with a pointer of its own

Its pointer is read only on a stopped machine, in units="screen", in a mode of known size; otherwise pointer.program says why it was not read. A drag's end is not corrected. When the target is past what the driver's range lets that pointer reach, the server widens the range once; range_widened then shows in every reply's session.

Allegro's default DOS mouse driver keeps its own pointer this way. With mouse = I33 under [mouse] in the program's allegro.cfg it reads the driver instead, sets the range to its screen, and is pointed at by pixel like any other program.

Buttons

  • double and triple are two and three clicks a timer tick apart, close enough for a program that counts them.
  • A button with no x, y acts where the pointer is. A move and a button in one call move first.
  • A cancelled call does not let go of a button it pressed, since action="down" holds one on purpose.
  • A call with a button and no move answers arrived: true.

A click that fires a code breakpoint ends the call wait_stopped, stopped machine or running: the program reacts within a frame, before the pointer is read back. In a perform that step is step_failed. A read-back that fails after the commands went in is refused with delivered: true, so nothing is sent twice.

The rest of the reply

With target, the reply adds target, screen (the screen recognised) and identified_at_stop, the number of the stop at which screen_map_find recognised it.

In a perform run with summaries, each frame a send ran on a stopped machine is a numbered stop, and only the last one is in a reply. stops.unaccounted counts the stops numbered during the list that no step's reply shows, so a few sends make it large.

The helper, a stopped machine, expect and a failed send work as under send_keys. With no logfile the mouse goes in unchecked too.

The pointer you see

With a window and its border, the first move also shows an orange pointer: the DOS mouse driver's default graphics pointer, one cell per screen pixel. In a graphics mode it is the size of the program's own pointer, whatever the window's size. The mouse driver's pointer is at its new place at once; the orange one glides there briefly. It is never in a capture, and a failure to draw it fails nothing.

The contract your client receives
Text
Move the guest's mouse pointer, press buttons, and read the pointer back; `expect`
proves the program reacted, as in `send_keys`, and is waited up to `seconds`.

`x`, `y`: both or neither, in `units`. `action` names what the button does; with an `action`
and no `button` the left one is used, and with neither only the pointer moves. `target`: a
control of the latest `screen_map_find`
screen, instead of `x`, `y`. `action="drag"` presses at that place and releases at `to_x`,
`to_y`, with the move between. A stopped machine is run as for `send_keys`; under a map that
declares the program's own pointer, a move there nudges the driver until that pointer is on
the place and `arrived` judges it (`pointer.program`, unavailable with its reason on a running
guest, `units="mouse"` or a mode of unknown size). A drag's end is not steered.

Refused: no move and no button, half a move, a place off the screen, no stub, a stopped
machine with no `RUNSTATE`; `target` beside `x`, `y` or `units="mouse"`, a drag without
both ends, `to_x` and `to_y` without a drag (`bad_argument`);
no `screen_map_find` match (`needs_call`); an unknown control (`unknown_ref`).

Reply: `route`, `pointer`, `arrived` (the program's pointer read, or `units="mouse"`; else
`driver_arrived` and `asked`: only the driver is known), `motion` (a move reached the motion
counters too), `verify`; with `target`: `target`, `screen`, `identified_at_stop`; from a
stopped machine, `send_keys`' `polls` and stop.

Reference: REFERENCE.md, Input and the screen.
Tools