REBECCA UMBACH, NICOLA HENRY, and RENEE SHELBY

We have two main goals in attending this workshop. First, the policy and Trust & Safety teams of many technology platforms around the world are focused on the role user-generated content created through generative AI tools may play in impacting elections, helping propagate misinformation, and scale financial fraud operations. While important work, this narrow focus misses other harmful forms of user-generated content, including IBSA. It is crucial to recognize the significant harms associated with AI-generated IBSA, which at first glance, may feel less critical. Moreover, effective interventions require multi-disciplinary collaboration among ML researchers, engineers, policy teams, and scholarly experts. Second, our goal in attending this workshop is to better understand how other researchers explain complicated concepts to research participants, and identify angles we may have missed in our prospective research plans. Below, we briefly detail the methods and results of our original research, concluding with a brief summary of our prospective survey format.

BRENNAN ANTONE and MALTE JUNG

Online communities whose topics focus on Generative AI (Gen-AI) serves as venues for learning and sharing content from Gen-AI tools. Employing a case study approach, this work examines theMidjourney communities on Discord, Facebook, and Reddit. On each platform, affordances and boundary regulation shape community interaction. AI users may come into contact with AI critics or less engaged members of the lay public on each platform. Moderators face distinct challenges on each platform, which underscores the importance of community management for Gen-AI discussions online.

SABILA NAWSHIN, ALICIA NICOLE FREEL, MINJE KIM, and APU KAPADIA

The widespread availability of deepfake technologies presents unprecedented challenges in addressing the spread of misinformation. With the emergence of Large Language Models (LLMs) capable of generating realistic and convincing deepfakes, the possibility of generating audio and video content that can be used to subtly manipulate people into believing and spreading misinformation is now a realistic threat. Our work aims to study the subtle manipulation of people using familiar voices in deepfake based misinformation. In particular, we intend to find the effects of familiarity in the believability of information. The primary focus for this work is on the believability of misinformation conveyed by synthetic voices that contain voice characteristics of popular celebrities whose voices are found to be familiar to young adults belonging to Generation Z (aged between 18 to 26). A primary quantitative study has been conducted within the United States (N = 110) to identify celebrity voices familiar to the Generation Z population. With the top 6 celebrities identified, another quantitative experiment was conducted (N = 102) to identify deepfake voice clones of the selected celebrities that retain a varying degree of familiarity present in the original voice. This is a work in progress, and we aim to study the susceptibility of the general population, children, and older adults to such familiar deepfakes.

JIANNAN LI

Generative artificial intelligence is making profound impact on multimedia content creation and consumption. This position paper discusses a potential future model of these activities, where multimodal intelligent agents select, recommend, and generate content on behalf of content creators in response to consumer input and behaviors. We identify technical challenges for building and evaluating such agents, design considerations for user interfaces and agent behaviors, and social implications for content creators. We hope this piece can open up conversations about one possible path of user generated content with the tempting goal of maximizing scalability and adaptability.

YUCHENG JIN

Generative AI has emerged as a powerful tool with the potential to complement human creativity and support mental well-being. This abstract aims to investigate the applications of generative AI within therapeutic contexts, particularly focusing on therapies involving creative activities (e.g., music therapy and art therapy). Our recent research has explored the role of generative AI in enhancing the practice of therapists in music therapy sessions and its utility in facilitating music-based reminiscence activities for older adults. The proliferation of therapeutic content on video platforms such as TikTok, YouTube, and Bilibili, including Autonomous Sensory Meridian Response (ASMR), music therapy sessions, and mindfulness exercises, has created a new ecosystem where creators are constantly seeking innovative ways to engage their audience. Generative AI stands at the forefront of this creative quest, offering content creators a suite of tools to foster enhanced creativity and productivity. Therefore, I am interested in discussing the following topics in this workshop.

CHARLOTTE BIRD

Generative AI (gen-AI) tool offers unprecedented capabilities for creating content. Images, text and video are the most popular applications of gen-AI, with models such as StableDiffusion and ChatGPT amassing thousands of users. Innovation does not look set to slow down, with the recent release of impressive text-to-video model, Sora. Understandably, the rapid adoption of new gen-AI tools has precipitated a number of concerns, such as the rise of deepfakes, fake news, copyright infringement and job displacement. However, with adoption of gen-AI technologies only looking to increase, it is essential to foster safe, useful and fruitful human-AI interactions. User-generated content is a prime example of such interaction. UGC — any content made by a user and not a brand — dominates social media sites as other users seek authentic, credible, relevant and relatable content. Gen-AI offers opportunities for users to generate ideas, images, scripts and captions, among countless other applications.

ZIYI ZHAO

The phenomenon of deepfake, leveraging generative AI to create hyper-realistic videos and audio recordings, presents a formidable challenge in the digital age, blurring the lines between reality and fabrication with implications for privacy, security, and trust. This study addresses the critical research question of how deepfake technology affects the detection of cybercrime, particularly through the impersonation of individuals. Grounded in Interpersonal Deception Theory (IDT), the research modiRies traditional constructs to encompass the unique characteristics of deepfakes. The empirical investigation faces hurdles in acquiring authentic deepfake data and ethical concerns, addressed through a mixed-method approach including sentiment analysis of online videos, controlled experiments, and semi-structured interviews. Preliminary Rindings indicate varied public perceptions of deepfakes, inRluenced signiRicantly by the content's context. This research contributes to the HCI Rield by elucidating the dynamic interaction between deepfake technology and human cognition, offering insights for developing more effective cybercrime detection and prevention strategies.

HUIRAN YI and LU XIAN

Generative AI (Gen-AI) has come to the center of attention in the advertisement industry worldwide. Marketers and influencers are key actors who celebrate the most up-to-date Gen-AI technologies. The reception of Gen-AI is mixed: it is heralded as a creative and liberating tool for generating innovative and captivating content ideas, yet there is a concurrent concern among advertising professionals about the potential for technology to replace human labor.
Our analysis centers the empirical cases from Xiaohongshu (RED), a prominent social media platform in China that merges social media functionalities with e-commerce. RED, known for its emphasis on text-, image-, and video-based content, mirrors the influencer marketing practices found on global platforms like Instagram and earns over 80% of revenues through advertisement. Among existing usage of Gen-AI for marketing purposes, we focus on a particular kind of content generation: the content generated by social media influencers.We specifically examine “Key Opinion Consumers (KOCs)”—ordinary platform users who carry advertorial content on RED. Given their authenticity and relatability to everyday consumers, they are popular advertisement agents for commercial brands. KOC marketing campaign on the RED platform is an established business model in the industry. Our previous work (under review) has pointed out that integrating KOCs into marketing campaigns requires labor from multiple actors, including KOC themselves, marketers, and talent brokers (i.e. multiple-channel-network agencies).

ETHAN HAQUE, MOHAMED ABDELNABY, JOHANA ARGUETA, CINDY LE, and NHU VAN

In this position paper, we explore the impacts of deepfake technology, a product of Generative Artificial Intelligence (Gen-AI), on Generation Z’s well-being. With emphasis on mental health implications, potential for sexual exploitation, as well as spread of misinformation, this paper argues that while deepfakes have many innovative benefits in the world of content creation, there are some unethical usage that poses significant risks to society, which calls for action strategies to highlight and harness the benefits of Gen-AI while safeguarding its adverse effects.

TRAVIS LLOYD

As generative AI tools gain popularity, online communities are forced to adapt their policies and practices to deal with the arrival of AI-generated content (AIGC). While some communities may welcome this new form of participation, others will view it as an existential threat. Our prior work interviewing Reddit moderators suggests that AIGC is uniquely threatening to communities who value authenticity and factuality. These communities may enact rules banning AIGC, but without foolproof detection tools, enforcement by human moderators is challenging and relies on flawed heuristics. In future work we would like to explore design interventions that can help moderators from these communities adapt to this new reality. Additionally, for communities that welcome AIGC, we would like to study how the presence of AIGC impacts discussion dynamics, user participation, and community trust. We hope to complement our qualitative methods with computational techniques that will enable platform-wide measurements of the impacts of AIGC. Reddit’s distributed self-governance model makes it an ideal site of study for HCI researchers who are interested in studying how communities adapt to technological change. We hope to eventually perform similar studies on different platforms that take a top-down governance approach, such as StackOverflow, to see what we can learn from platforms with different governance paradigms.

YU-LING CHOU, XUYUAN CAI, and CHIA-FANG CHUNG

The prevalent use of social media has increased attention on the content created by content creators. Previous academic research has investigated the factors that foster audience engagement with content creators, focusing on concepts like parasocial and trans-parasocial interaction. Similarly, our research aligns with these findings, highlighting that reciprocal interaction plays a crucial role in bolstering audience engagement. This dynamic is particularly evident in the context of live streaming, where streamers enhance engagement through direct interactions, such as real-time comments, collaboratively shaping the content and atmosphere.
However, research on the auditory experience within the context of live streaming is limited, despite its proven effectiveness in enhancing emotional engagement across various domains, such as gaming. Particularly, the potential of live streams’ interactive nature to enrich the auditory experience of viewers remains underexplored. Existing studies on auditory experience have predominantly focused on background music and its static and non-interactive attributes, neglecting the interactive opportunities in live streaming. In recent times, the significance of background music in content creation has begun to attract academic interest. Background music is commonly used to complement visual content in nonstreaming videos, and platforms like TikTok, Instagram, and YouTube have facilitated the integration of music into the content creation workflow, simplifying the process of adding background music while also enhancing user convenience. Exploring the use of auditory experiences to increase viewer engagement presents a compelling opportunity for research into how live streaming can adjust various musical elements to support different interaction contexts.

DEREK HANSEN, JERSON FRANCIA, MATTEW TAYLOR, SHYDRA MURRAY, and BEN SCHOOLEY

Our research team at BYU has been studying the capabilities of Gen-AI in creating spear phishing messages, as compared to trained humans. We have employed a novel approach, wherein we recruited “targets” who share personal information about themselves (e.g., job title and location; personal hobby; something they have posted about online) and agree to return later for an interview. We then have humans who have been trained on how to create effective spear phishing messages create them based on the information provided by the targets. We also have Gen-AI (i.e, ChatGPT-4) generate spear phishing messages using the same prompts we gave the humans. Finally, we invite the targets back and show them 12 spear phishing messages created to target them based on the information they provided. They are asked to “sort” the messages from most compelling to least compelling and discuss why they placed them where they did. After the sorting and discussion, they are told that one or more of the messages were created by AI. They are asked to place a token (with the letters AI) on any they believe were created by AI and explain why they chose those ones and not others. We are currently analyzing the data for 28 targets. Preliminary results suggest that Gen-AI is slightly more effective at creating personalized spear phishing messages than trained humans. And this is based on an off-the-shelf Large Language Model (LLM) and a very basic prompt (“Create a spear phishing…”). Furthermore, most targets have no idea which messages were created by AI and don’t have an accurate mental model of how to even approach that question. Through this project and future projects on the creation of disinformation messages, images, and videos in the form of hypothetical social media posts, we hope to demonstrate the capabilities (and potentially limitations) of Gen-AI in personalizing disinformation. This is a necessary step in order to identify and Olag such content so the risks associated with fraud and disinformation can be mitigated.

Llogari Casas, Kenny Mitchell, Monica Tamariz, Samantha Hannah, David Sinclair, Babis Koniaris, and Jessie Kennedy

We consider practical and social considerations of collaborating verbally with colleagues and friends, not confined by physical distance, but through seamless networked telepresence to interactively create shared virtual dance environments. In response to speech recognition textual language prompts our, HoloJig system performs according to Dolgoff and Roddenberry’s science fiction language guided Holodeck narrative environment generation. Here instead realized for presentation through virtual reality headsets to create dance halls, clubs, concerts or more abstract spaces.

SANA HASSAN IMAM, CHRISTOPHER A. METZ, and ROLF DRECHSLER

Generative AI is reshaping the boundaries of creativity and productivity across every field. Idea crowdsourcing platforms rely on user creative engagement to solve problems, innovate, and co-create value through the collective intelligence of individuals. Traditional idea-generation methods may limit idea novelty and diversity and require significant human creativity. Boosting creativity is an imperative requirement of the idea crowdsourcing platforms, where creativity is a complex phenomenon to measure. To solve this problem, we propose a generative AI model that works in two steps: 1) evaluates the creativity scores of users’ ideas through the uniqueness, diversity, and feasibility scores calculated using the idea pool and peer and manager’s feedback and 2) provides prompts designs to stimulate, enhance and refine the idea’s quality of individual users based on the calculated creativity scores. We propose a framework for future research into creativity enhancement through generative AI in idea crowdsourcing platforms. This framework can help users enhance their creative contributions and help platform administrators identify high-potential ideas that may not receive high votes.

YONGNAM JUNG, JIAQI (AGNES) BAO, and PEIXIN HUA

As AI-generated figures, photos, and videos become increasingly common, individuals struggle to discern genuine content from fabricated ones. With younger generations growing up surrounded by media saturated with easily created fake content, the concepts of ’authenticity’ have transformed. Consequently, alongside ’credibility’ and ’trustworthiness’, authenticity emerges as a crucial concept for comprehending the consumers’ perception in the contemporary media landscape. Authenticity pertains to staying true to oneself. Authenticity is not regarded as an inherent quality of an object but rather as a reflection of one’s own beliefs, expectations, and perspectives. Thus, authenticity can be built in even a fabricated context if it meets users’ expectations. In short, authenticity refers to an entity’s ability to align with users’ expectations of a platform or content by creating a relevant schematic fit. Although AI-generated content was once considered fake, our understanding of how this perception is evolving remains limited, particularly as users become more knowledgeable about the uses of generative AI in content creation. We aim to explore how AI-generated content affects users’ perception of authenticity on social media.

ZIMO XIA, XICHEN SUN, XUEYANG WU, QIYU PAN, KAIXIANG MO, and ZHENHUI PENG

In this paper, we provide insights to the question “what values can AI-generated content bring to social media users”. We present a technical prototype SummarFlex that support news filtering and reading with generated query-focused hierarchical summarization. First, we describe the motivation and design of SummarFlex. Then, we discuss the benefits and concerns of enabling users to customize the content in social media.

SHUNYI YEO and SIMON T. PERRAULT

This position paper explores the integration of Generative AI (Gen-AI) technologies, specifically Large Language Models (LLMs), to foster reflection among users in the context of User-Generated Content (UGC) on online platforms. By leveraging the capabilities of LLMs, this paper proposes a novel approach to organically induce reflection in users before they contribute to online spaces. This paper elucidates the mechanisms through which Gen-AI technologies can facilitate reflective engagement, going beyond the mere generation of information content. Ultimately, this paper advocates for the strategic integration of Gen-AI technologies to enhance the quality and depth of UGC on online platforms, thereby enriching the collective discourse and fostering a more reflective online community.