In order to be able to generate a rich picture of the Developer Experience of the product, we have developed a number of open-ended questions which will help inform how our test subjects have perceived the software. Beyond the standard quantitative measurements, we use the following tools to help provide a deeper understanding of issues and impressions that arise when working with a product.
| Data | Description |
|---|---|
| Debriefing Survey | The debriefing survey will tell us what stands out about the developer’s experience of using the product. This will help us set priorities for potential changes to the product in general. |
| Bugs and Errors | This is a listing of bugs and errors encountered when working with developer resources. |
| Verbal Protocol | This is the running commentary provided by participants as they think aloud. This provides us with indicators surrounding what may have confused participants and why. |
| Desktop Video | This describes the screen recordings taken of the participant’s primary display. |
| Code Samples | Here we provide a recording of any file changes made by participants within the working directory. |
| Application Usage | This comprises logs of any activity within the foreground application. It is sampled every minute during the duration of the test. |