Introduction

  • The purpose of this project is to turn a wired mouse connected to the raspberry board that takes advantage of the built-in Bluetooth functionality of the raspberry board and turns them into a wireless mouse/keyboard for the devices it is connected to. You see the model described here.

  • But the wireless mouse/keyboard is not the main content of this project. The nice part here is that you will be able to emulate a mouse or keyboard wirelessly without the need for raspberry attached hardware. Instead, we will use code to simulate keyboard and mouse signals and send them to the remote device (phone, computer …).
  • With code control, it is easy to build rich applications, for example:
    • With a Webcam attached to raspberry, you can automatically move, click, drag on another computer, this way to hack the game is not bad !!!
    • You really like a game on Window, Osx, iOS, assuming the racing game needs a keyboard. With this app, you can mount sensors.
  • The project should look like this

Who will need keyboard / mouse emulation functionality?

  • In addition to the “fun” projects described above, this function will be useful for those who need to develop a software testing tool related to UI. With the full simulation of mouse and keyboard actions, you can build test scripts automatically without having to spend time and people doing it again and again.
  • Those who do apple apps that are related to hardware, Bluetooth will have problems Apple strictly checks applications related to the hardware of iPhone, iPad … With the use of popular Bluetooth profiles. This you will not have to interact with Bluetooth hardware.

Project software design.

  • For flexibility in usage, I will build on a server-client model.
  • Also need to use utilities of Linux and Bluetooth Stack to control Bluetooth related tasks.
  • The design model here is in the form of multiple processes. The detailed functions of the processes are described below.
  • In order to perform this request, we need the support of the following components
    • hciconfig utilities: This is a utility to help configure Bluetooth drivers, such as selecting client or server mode, allowing scanning, changing advertise information.
    • Bluez stack: This is the famous Bluetooth stack on Linux, is open-source and we just go to config, not code, add. Responsible for roles such as active profile registration: audio, mouse, keyboard, printer … APIs are exported and dbus support.
    • Bluetooth server app: This is the first and important application that we need to build. It performs communication with the Bluez and sends the data to the Bluez to maintain communication. It also exports the API via dbus so that clients connecting to it can directly send data to Bluez.
    • Mouse client / emulate app: This is an app that has been subtracted from the Bluetooth domain, the app collects data and connects to the server app to send mouse data to Bluez.
    • Keyboard client/emulator app: This app is similar to the mouse client app, only with different raw data characteristics of different keyboard and mouse.

Bluez Stack

  • Bluez stack is used to configure Bluetooth profiles, for example, Keyboard and Mouse profiles in this project. In addition, you can also set your Bluetooth chip to work on other profiles such as SPP, GATT …
  • Route data is received from the driver to the user app, controlling the connection between the Remote device and the Raspberry.

Bluetooth server app

  • The server will act as the client for Bluez and the server role for the mouse client and keyboard client apps.
  • The server uses dbus path: / org /bluez to transfer data to the Bluez stack, Bluez communicates with the Bluetooth driver and transmits/receives data with the remote device.
  • Using dbus to communicate with bluez is a must. To receive data from the client, we can use any method, like shared memory, pipe, socket …
  • In this project, I always use dbus as a method of communicating with the clients. To create a dbus server, we need to register a dbus path with the system with access requirements.
dbus_config_file
  • In this project, I registered a path named thanhle.btkbservice, with root user rights. This means that only the root users can open the server on this path.
  • The following code in the server code does the registration of a system bus with dbus.
resister_bluetooth_mouse_on_pi_service
  • After registering to open the server, it is necessary to declare the method that the client after connecting to the server can call to push data to the server. Currently, the server supports two types of devices, Keyboard and Mouse. Therefore I will register 2 methods: send_keys and send_mouse.
register_dbus_method

Mouse client app

  • As described above, the server will handle the transmission of data to Bluez based on the specification of Bluez, so the client communicates with the server based on the custom dbus path. thanhle.btkbservice. completely independent of the system bus.
  • In the future, we can expand to other types of communication for example via the internet, MTQQ … Now the application can control Mouse and Keyboard via the internet.
  • Currently, the code is supporting 2 types of clients.
    • Type 1 (mouse_client): reads the raw data from the physical mouse connected to the Raspberry and sends the signal to the remote device (Phone, PC, Ipad…). In this way we turn a wired mouse on the PI board into the wireless mouse of the remote device.
    • Type 2 (mouse_emulate): uses code to simulate mouse movements and send the signal to the server. This approach is more useful for you to program automation applications without human manipulation.
  • The common feature of these two methods is the need to connect to the dbus server created by the server app.
bluetooth_mouse_client_init_dbus

The client-side work is much simpler. After connecting, just prepare the data and call the send_mouse method.

mouse_client_send_data
  • In the case of mouse_client, we need to add code to read the raw data from the device driver and send it to the server.
bluetooth_mouse_send_state

Keyboard client app.

  • The scope of this article only covers Mouse operation, if you are looking for information about Bluetooth keyboard emulation on raspberry, you can refer to the article HERE.

How to use?

Setup

    • Install git if you haven’t installed it before
 sudo apt-get install git
    • Clone source to raspberry boards
git clone https://github.com/quangthanh010290/keyboard_mouse_emulate_on_raspberry.git
    • Run the setup.sh file to install the necessary packages.
cd keyboard_mouse_emulate_on_raspberry
sudo ./setup.sh
    • Reboot board
sudo reboot

Run the application

  • Run the server application
  • If you have experience using tmux you can run the boot.sh file, by default the btk_server.py file will be called in a tmux windows, you can open more windows and run mouse or keyboard client applications.
  • cd keyboard_mouse_emulate_on_raspberry
    sudo ./boot.sh
    tmux a
  • If you do not know anything about tmux then you need to open different terminals and on each terminal, you will run server and client applications.
  • When the server is running successfully, your terminal screen should look like this:
wait_for_bluetooth_connection
  • Now you scan the Bluetooth on your phone will see the device name appear. You can connect to the Raspberry now.
  • The raspberry console screen will also show information about the Remote Device it is connected to.
  • At this point, the server app has established a remote Bluetooth device connection and is ready to receive data from the physical mouse to send to the remote device.
  • Next need to open one more terminal to run the client app.
  • Mouse_client will automatically detect the mouse on the system. Please note that you should only plug 1 mouse into the raspberry, as the current code is selecting the first mouse device it finds. If you have more than 1 device, it is not certain which mouse will be selected.
  • Here are the results on the phone.

Maybe you are interested.

  • Make Raspberry Pi become WiFi access point.
  • Become an Internet Service Provider with Raspberry Pi

Contact the author.

If you need advice or need to answer any questions, contact the author HERE