Measuring Mobile AIGC Quality Without User Studies or Sensors

Table of Links

Abstract and 1. Introduction

1.1 Background

1.2 Motivation

1.3 Our Work and Contributions and 1.4 Organization

2. Related Work

   2.1 Mobile AIGC and Its QoE Modeling

   2.2 Blockchain for Mobile Networks

3. Preliminaries

4. Prosecutor Design

   4.1 Architecture Overview

   4.2 Reputation Roll-up

   4.3 Duplex Transfer Channel

5. OS2a: Objective Service Assessment for Mobile AIGC

   5.1 Inspiration from DCM

   5.2 Objective Quality of the Service Process

   5.3 Subjective Experience of AIGC Outputs

6. OS2A on Prosecutor: Two-Phase Interaction for Mobile AIGC

   6.1 MASP Selection by Reputation

   6.2 Contract Theoretic Payment Scheme

7. Implementation and Evaluation

   7.1 Implementation and Experimental Setup

   7.2 Prosecutor Performance Evaluation

   7.3 Investigation of Functional Goals

   7.4 Security Analysis

8. Conclusion and References

2 RELATED WORK

In this section, we review some related works regarding mobile AIGC and its QoE modeling, as well as the progress of mobile-oriented blockchains.

2.1 Mobile AIGC and Its QoE Modeling

The success of AIGC relies on large pre-trained models with billions of parameters [1]. Obviously, such a paradigm is unsuitable for numerous resource-constrained clients, hindering the further development of AIGC. To this end, researchers have been reducing the AIGC model size and alleviating the hardware requirements. On Feb. 2023, Qualcomm AI published the world’s first on-device version of Stable Diffusion, one of the most famous text-to-image AIGC models [19]. By quantizing the model parameters from FP32 to INT8, the shrinking model can be smoothly operated on common smartphones. Likewise, Chen et al. [5] present a series of GPU-aware optimizations for Stable Diffusion, such as flash attention and Winograd Convolution, achieving the 12-second inference latency on Samsung S23 Ultra. SnapFusion [20] further reduces such latency to 2 seconds by step distillation. Nowadays, various mobile AIGC applications have been launched and widely used in practice, such as dreamer[1] and Draw Things[2]. In the foreseeable future, AIGC will further embrace mobile networks, enjoying the easy-accessible mobile communications, computation, caching, and personalization [6], [7].

The QoE modeling of mobile AIGC is intractable due to the unique features of AIGC outputs as artworks. Compared with conventional scenarios like crowdsourcing and edge caching, the QoE of mobile AIGC is greatly affected by implicit subjective user preferences. Traditionally, such subjective QoE can be measured using under study [21] or analytics frameworks, e.g., absolute category rating [9]. However, these methods are tedious and time-consuming, without the potential for real-time QoE evaluation. From another perspective, physiological methods aim at reflecting the perceived QoE of clients directly from their perceptual and cognitive processes, e.g., heart rate, blood pressure, and temperature [22]. Although first-hand feelings can be acquired, clients should keep wearing specific monitors, which might be invasive and unsuitable for large-scale applications. Recently, another emerging technique called affective computing gained significant attention in this field. Affective computing can measure the human perceived QoE according to the affective behaviors driven by human emotions, e.g., facial expressions, speech tone, and body gestures [10], [11]. Nevertheless, training the neural networks for personalized affective behavior analysis requires tremendous time and computation resources. Furthermore, the resulting model is modality-specific because the types of affective behaviors that clients play in different scenarios differ. Apart from measuring implicit subjective QoE, another issue is fusing it with objective service qualities. Du et al. [7] adopt Weber–Fechner Law [23], which can evaluate the change of client perceivable experience caused by changing service qualities. However, they apply the unified model for all clients while ignoring their heterogeneity in terms of personalized preference, strictness, etc.

Insights: In this paper, we establish a reputation scheme to measure the subjective experience of clients. As shown in TABLE 1, our approach outperforms existing works in two aspects. Firstly, the clients can adopt personal QoE models and express their perceivable experiences in a uniform form of opinions. Note that the opinions of individual clients may contain subjectivity and bias. Hence, WSML theory is adopted to alleviate the influence of these factors and provide each MASP with a fair reputation. As a result, the timeconsuming user study, physical tests, and numeral network training can be circumvented. Moreover, the existing QoE measurements are modality-specific since the user study, tests, and training objectives vary in different modalities. In contrast, the reputation approach can be applied to any modality.

1. https://stablediffusionweb.com/app

2. https://drawthings.ai/