13. User Interaction & Gesture Recognition

When the user interacts with the display of a device, many gestures are generated, e.g. scroll. Android handles touch events using onTouchEvent() method.

The gesture starts when the user first touches the screen, continues as the system tracks the position of the user’s finger(s), and ends by capturing the final event of the user’s fingers leaving the screen. Throughout this interaction, the MotionEvent delivered to onTouchEvent() provides the details of every interaction.

13.1. Detect Common Gestures

Android provides the GestureDetector class for detecting common gestures. Some of the gestures it supports include onDown(), onLongPress(), onFling()

  1. Create a new empty project, and remove the Hello World TextView in activity_main.xml.

  2. Declare a class which implements GestureDetector.OnGestureListener interface including all callback methods. This can be a new class, or the within the activity class. GestureDetector.OnDoubleTapListener interface and onDoubleTap() method need to be implemented as well if double tap gesture detection is required. For example:

    class MainActivity : AppCompatActivity(), GestureDetector.OnGestureListener, GestureDetector.OnDoubleTapListener {
    
      override fun onFling(event1: MotionEvent, event2: MotionEvent,
                           velocityX: Float, velocityY: Float): Boolean {
          return true
      }
    
      override fun onScroll(e1: MotionEvent, e2: MotionEvent,
                            distanceX: Float, distanceY: Float): Boolean {
          return true
      }
    
      override fun onLongPress(event: MotionEvent) {
      }
    
      override fun onShowPress(event: MotionEvent) {
      }
    
      override fun onSingleTapUp(event: MotionEvent): Boolean {
          return true
      }
    
      override fun onDoubleTap(event: MotionEvent): Boolean {
          return true
      }
    
      override fun onDoubleTapEvent(event: MotionEvent): Boolean {
          return true
      }
    
      override fun onSingleTapConfirmed(event: MotionEvent): Boolean {
          return true
      }
    
      override fun onDown(event: MotionEvent): Boolean {
          return true
      }
    
  3. Instantiate a GestureDetectorCompat object.

    private lateinit var gesDetector: GestureDetectorCompat
    

    Then instantiate the gesture detector with the application context and an implementation of GestureDetector.OnGestureListener. And set the gesture detector as the double tap listener.

     override fun onCreate(savedInstanceState: Bundle?) {
        ...
        gesDetector = GestureDetectorCompat(this, this)
        gesDetector.setOnDoubleTapListener(this)
    }
    
  4. Now implement the onTouchEvent() Method. To make it possible for GestureDetector object to receive events, override onTouchEvent() method, and pass along all observed events to the detector instance.

    override fun onTouchEvent(event: MotionEvent): Boolean {
       this.gesDetector.onTouchEvent(event)
       return super.onTouchEvent(event)
    }
    
  5. Add a Toast for each callback method. Hence when running the app, user gesture and the corresponding triggered method will be displayed. Example code:

    Toast.makeText(applicationContext, "onFling", Toast.LENGTH_SHORT).show()
    

13.2. Multi-touch Gestures

Multi-touch gesture happens when more then one finger touches the screen at the same time. Android allows us to detect these gestures and following touch events will be generated:

  1. ACTION_DOWN: For the first pointer that touches the screen. This starts the gesture.

  2. ACTION_POINTER_DOWN: For extra pointers that enter the screen beyond the first.

  3. ACTION_MOVE: A change has happened during a press gesture.

  4. ACTION_POINTER_UP: Sent when a non-primary pointer goes up.

  5. ACTION_UP: Sent when the last pointer leaves the screen.

In order to identify the index of the pointer that triggered the event, use the getPointerId() method to obtain a pointer’s ID to track the pointer across all subsequent motion events in a gesture.

  1. In order to detect any of the above event , onTouchEvent() needs to be modified, since it handles touch screen motion events.

    First it should identify how many pointers are currently active on the view:

    val pointerCount= event.pointerCount
    
  2. Then, a set of tasks need to be performed for each active pointer: the X and Y coordinates of the touch, the corresponding event ID, action type, action index, and a string variable.

    for (i in 0 until pointerCount){
    
      val x = event.getX(i)
      val y = event.getY(i)
      val id = event.getPointerId(i)
      val action = event.actionMasked
      val actionIndex = event.actionIndex
      var actionString: String
    }
    
  3. Action types equate to integer values, use when to covert action a string description.

    when (action){
    
       MotionEvent.ACTION_DOWN -> actionString = "Down"
       MotionEvent.ACTION_MOVE -> actionString = "Move"
       MotionEvent.ACTION_POINTER_DOWN -> actionString = "Pointer Down"
       MotionEvent.ACTION_UP -> actionString = "Up"
       MotionEvent.ACTION_POINTER_UP -> actionString = "Pointer Up"
       MotionEvent.ACTION_OUTSIDE -> actionString = "Outside"
       MotionEvent.ACTION_CANCEL -> actionString = "Cancel"
       else -> actionString = ""
     }
    
  4. Next, in activity_main.xml, create two TextView objects. One constraints to top-right corner and name it “textView1”, the other one constraints to top-Right corner and name it “textView2”.

  5. Now, create a string message containing actionString value, action index, pointer ID, and X and Y coordinates. The ID value is then used to decide which TextView object the string should be displayed:

    val touchStatus ="Action: $actionString Index: $actionIndex ID: $id X: $x Y: $y"
    
    if (id == 0)
        textView1.text = touchStatus
    else
        textView2.text = touchStatus
    
  6. Now install and run the app. It should look like the pictures below on the Lenovo tablet.

    ../_images/gesture_screenshot.jpg