Inappropriate online discussion is a problem, and a multi-year research project addressed it from the perspective of emotional psychology. I worked on the project as a designer and doctoral researcher. I will now highlight key points emphasizing the multi-phase process that led to the presentation of this video prototype.

My Task

In the research project, my task was to explore possible interface solutions to address inappropriate news commenting using speculative and critical design methods. Specifically, solutions that would address inappropriate comments by supporting users' emotional regulation. My ideas had to be promising from a psychological perspective and serve as discussion starters in user research. The reason for this was that very little was known about the possibilities and risks of addressing inappropriate commenting, and information was sought from both journalists and commenters.

Idea Generation and Selection for Further Development

I started by sketching about 60 ideas on paper and discussing them with my supervisor, user-centered design professor Thomas Olsson, and postdoctoral researcher Heli Väätäjä.

I evaluated all my ideas based on feasibility, novelty, and difficulty, and discarded about 20 ideas. Of the remaining 40 ideas, 19 proved promising, such as a virtual audience for online news commenters. I created more detailed wireframes using Balsamiq and presented the ideas to the research team, which included the aforementioned individuals as well as three psychologists and an IT expert. The psychologists provided feedback that some ideas contained too many gamified elements or were unclear. Regarding the audience idea, the psychologists noted that it could concretely illustrate the emotions a text might evoke, and they found it promising.

I considered the psychologists' feedback and selected the audience idea and five other top ideas from the wireframes to present at a meeting with journalists and media researchers from Rajapinta Ry. I asked for feedback on the ideas from an audience of about twenty people via a short form. Based on the feedback from journalists and media researchers, the virtual audience seemed promising and thought-provoking, and it would thus be suitable for opening up discussions with research participants.

Wireframes as Part of Interview Research

The six wireframes I selected for solutions to address inappropriate news comments were part of interviews with 10 experienced Finnish journalists. The interviewees were asked to think aloud about the ideas presented to them, regarding their feasibility, etc. The results were published at CHI, the flagship conference in the research field.

The interviews revealed, among other things, that the idea of a virtual audience for news commenters was seen by experienced journalists as questionable, but sensible. Thus, I deemed it worth further research and planned lab experiments with prototypes.

Coronavirus and Online Survey for the Ideas

Due to coronavirus restrictions, all lab experiment plans had to be canceled, and since it was too early to test the ideas live online, the research on the ideas had to be conducted via a survey. With the support of my colleagues, I conducted an international survey for online news commenters regarding the proposed methods for addressing inappropriate commenting. I created more refined versions of eight ideas, including the audience idea, using AdobeXD and built illustrative storyboards for the survey.

In the survey I designed, each respondent evaluated two randomly selected ideas and then chose and explained which one was better. This setup produced two types of data: which proposed ideas were considered better than others (quantitative results), and what aspects were noted in the evaluations (qualitative results). The qualitative results were published in the high-ranking BIT journal, and the quantitative results at the mid-tier domestic Academic MindTrek conference.

(The accompanying image shows the quantitative results. Lines and stars indicate significant differences. One star means there is a 95% probability that the difference is significant, two stars 99%, and three stars 99.9%.)

Based on the quantitative results, the idea of a virtual audience was rated less effective in improving the behavior of inappropriate commenters compared to many other proposed ideas (see the left side of the accompanying image). The audience idea was also rated as more inappropriate than many other ideas, although most respondents did not find it particularly objectionable (see the right side of the accompanying image).

Qualitatively, it became clear that providing feedback to the commenter about their writing via a virtual audience could make the commenter feel anxious or annoyed. For example, one respondent said: "I don't want to know that I’m being judged before my comment is even posted." On the other hand, many mentioned they would find the feedback helpful when writing.

Based on the conflicting results of the survey, I decided to create a version of the virtual audience that would only pop up when the user clicked 'send' on an inappropriate comment.

Second Online Survey

I conducted a second international online survey for news commenters, in which the new version of the audience idea and a standard 'text-based feedback window' were evaluated as a comparison (see the accompanying image "text-based alert"). The survey also included a variable for how much instructional text was given in the feedback (see the accompanying image "More1, More2, Less").

According to the survey results, a virtual audience popping up when sending an inappropriate comment was no more annoying to users than a simple text box (see the left side of the accompanying image). There was also no significant difference in how likely users believed it would change the commenter’s behavior for the better (see the right side of the accompanying image).

Additionally, the survey revealed that what matters in terms of whether feedback is believed to work is how precise the information it contains is. Vague feedback like "your comment might need adjustment" was not seen as effective as feedback like "this is how the audience might react to your comment," which included a virtual audience.

Thus, the idea of a virtual audience still appeared viable.

Video Prototype with AdobeXD

After the research project, I created a video prototype of the previous version of the audience prototype because I was curious about how the idea of a constantly observing audience could practically function. With video prototyping, I was able to experiment with what kind of audience gestures would be appropriate. For example, I realized that hand clapping or other rapid movements would irritate me as a writer. I also wanted to showcase my skills in my portfolio with the prototype.

I added a reference to a fictional "VirneGPT" AI model in the video prototype, as current AI models can easily simulate audience reactions and emotional responses from different personas. However, there are still many unanswered questions, such as "Does the graphical style of the audience matter much—for example, could AI generate realistic faces?", "How important is the accuracy of emotional reaction simulation?", and "Would a virtual audience, which takes up a relatively large amount of screen space, be practical in real use?"