ReadCtrl: Personalizing text generation with readability-controlled instruction learning (2024)

1 Introduction

Existing personalization methods mainly focus on the semantics of the content that individuals need, such as retrieving information based on individuals’ search queriesChen etal. (2023); Kirk etal. (2024); Shanahan etal. (2023) and summarization based on content representationRichardson etal. (2023).However, one important aspect of personalization that has not been widely explored is readability-controlled content generationVajjala (2021).This involves tailoring content to match individuals’ readability levels, which can vary widely due to differences in education and related experiences and trainingRibeiro etal. (2023).The emergence of large language models (LLMs) has further advanced this field, ushering in a transformative era of automatic content generationPu and Demberg (2023).It is crucial for content generated by these models to be accurate, relevant, and consistent with the cognitive abilities of the target audience.The emphasis on customized content creation underscores the critical role of customization and personalization in digital interactions, especially in environments where explicit instructions (typically from LLMs’ users) must be strictly followedZhou etal. (2023); Sun etal. (2023); Qin etal. (2024).At the heart of this innovative area are the principles of readability control instructions, designed to dynamically adapt the output vocabulary distribution to the specific context of each interaction.This can be achieved by analyzing chat history, interpreting user profiles, or responding to direct interaction requests, significantly enhancing LLMs’ versatilityHuang etal. (2023).

Previous efforts in the domain of controllable text generation have primarily centered on binary readability adjustments, typically categorized into tasks of simplification, complication, or sibling style transferGoldsack etal. (2022); Guo etal. (2021); Luo etal. (2022).Despite their objectives, these approaches often fail to fully address the diverse personalization needs due to the limited variety in training data and a concentrated emphasis on readability.In traditional supervised fine-tuning scenarios, designing multiple readability-level ground truths for training data to facilitate readability control is not scalable.As a result, models may not have sufficient exposure to varied text complexities, limiting their ability to adjust content according to user-specific readability needs accurately.In response, the field of artificial intelligence is shifting towards more dynamic systems that can adapt outputs to meet users’ unique preferences and requirementsKirk etal. (2024).This shift is heralding a new era of personalized content creation that extends beyond standard domains like information retrieval to specialized areas, enhancing user engagement and satisfaction through tailored content.

This paper addresses these challenges by introducing a novel methodology termed “readability-controlled instruction learning (ReadCtrl).”Our findings demonstrate that ReadCtrl can empower LLMs to transform input text into content with controlled readability accurately.As illustrated in Figure1, our investigation across a range of state-of-the-art LLMs shows varying degrees of compliance with readability-controlled instructions.Mainstream models like GPTAchiam etal. (2023) and ClaudeAnthropic (2024), despite demonstrating an Upward trend, fall significantly short of the ideal ‘perfect’ adherence curve—they show a tendency towards compliance but lack precise output control.In contrast, models that previously struggled with readability control, such as Mistral-7bJiang etal. (2023)—illustrated almost as a horizontal line in the figure—have shown significant enhancement with the integration of ReadCtrl, such as Mistral-ReadCtrl.These models now nearly meet the ideal performance curve, showcasing their improved ability to customize outputs to specific readability demands.Specifically, ReadCtrl incorporates explicit instruction tuning based on readability and is rigorously tested through tasks designed to evaluate the model’s ability to adjust output complexity.These tasks include text simplification, which aims to reduce the output’s readability relative to the input; paraphrase generation, which maintains the input’s readability; and semantic entailment generation, which adjusts the output’s readability, potentially increasing or decreasing it in relation to the input.By employing a clear instruction—“Given an input text, please output an entailment with a readability score around {target readability score}”—models like Mistral-ReadCtrl demonstrate the efficacy of ReadCtrl in generating content that is not only semantically coherent but also finely adjusted to meet diverse comprehension needs and preferences.

In our initial experiments, we evaluated the model’s performance in a "seen setting," where models were tested using datasets on which they were trained, such as ASSETAlva-Manchego etal. (2020) for text simplification, PAWSZhang etal. (2019) for paraphrase generation, and SNLIBowman etal. (2015) for semantic entailment. This setting established a baseline for instruction tuning, confirming that the models could effectively adhere to readability instructions in familiar contexts. Subsequent experiments involved an "unseen setting," testing the models against new datasets they had not encountered during training, such as WikiSmallZhu etal. (2010) for text simplification, MRPCDolan and Brockett (2005) for paraphrase generation, and MultiNLIWilliams etal. (2017) for semantic entailment. This phase was critical for assessing the models’ adaptability and accuracy in novel contexts, reflecting their generalizability and real-world applicability.We utilized several readability assessment metrics, including the Gunning Fog IndexGunning (1952) and Flesch-Kincaid Grade LevelKincaid etal. (1975), to quantitatively evaluate the complexity of the generated texts. Additionally, we employed a range of automatic generation metrics for generation quality evaluation, such as BLEUPapineni etal. (2002), SARIXu etal. (2016), FactualityLaban etal. (2022), Consistency and CoherenceZhong etal. (2022), to assess the quality of the generated texts, aiming to balance readability, information retention, factuality, consistency, and coherence during evaluation.

These evaluations confirmed the effectiveness of our ReadCtrl methodology across a diverse range of tasks and datasets. Particularly, Mistral-ReadCtrl excelled in both seen and unseen settings, showcasing robust performance metrics. For instance, in the unseen MRPC dataset, Mistral ReadCtrl achieved the lowest readability gap (1.66), the highest factuality (0.8184), and excellent BLEU (0.3798) and SARI (44.4327) scores, significantly outperforming competitors like GPT-4 and Claude-3. In the WikiSmall dataset, it recorded a readability gap of just 2.09, the highest coherence score (0.9763), and a strong SARI score of 42.1033.Furthermore, detailed human and LLM-as-a-judgeLan etal. (2024) evaluations were conducted to compare Mistral-ReadCtrl with GPT-4 across different tasks and readability requirements.Mistral-ReadCtrl demonstrated superior performance, achieving a win rate of 52.1% in human evaluations and 58.3% in AI assessments, compared to GPT-4’s 35.7% and 38.4%, respectively.Notably strong results were observed in tasks involving WikiSmall (62.5% in Human Eval, 67.7% in AI Eval) and SNLI (66.7% in Human Eval).

2 Methodology

3 Experiments

3.4 Results

3.4.1 Performance on seen tasks

Table 1 presents a performance comparison of Claude-3, GPT-3.5, GPT-4, and our model, Mistral-ReadCtrl, on seen tasks involving three datasets where instruction tuning was implemented: ASSET, SNLI, and PAWS. Regarding the Readability Gap, Mistral-ReadCtrl demonstrates superior precision in adhering to target readability levels, achieving the lowest scores across all datasets, indicating effective control over text readability. Factuality scores, which assess the accuracy of content compared to the original, show that Mistral-ReadCtrl maintains higher factual consistency than its counterparts. When evaluating Consistency and Coherence, which measure the logical flow and structural soundness of texts, Mistral-ReadCtrl performs robustly, though it is slightly outperformed by GPT-4 in the PAWS dataset. For BLEU and SARI metrics, critical for evaluating the linguistic and contextual appropriateness of text simplification and paraphrase generation, Mistral-ReadCtrl again posts the highest scores, showcasing its efficacy in producing high-quality, contextually appropriate responses.

3.4.2 Performance on unseen tasks

Table 1 illustrates the performance of four models — Claude-3, GPT-3.5, GPT-4, and Mistral-ReadCtrl — on unseen tasks, using the datasets WikiSmall for text simplification, MultiNLI for semantic entailment generation, and MRPC for paraphrase generation. These results are crucial for assessing each model’s ability to generalize beyond the data types encountered during training.

In the WikiSmall dataset, Mistral-ReadCtrl outperforms other models with the lowest readability gap of 2.09, suggesting superior control aligning with the target readability levels. It also achieves the highest factuality and coherence scores and significantly outstrips the competition in BLEU and SARI scores, indicating its effectiveness in maintaining content quality in text simplification tasks.

Mistral-ReadCtrl again shows notable performance for the MultiNLI dataset, which focuses on semantic entailment generation, with the lowest readability gap of 2.21 and the highest factuality and coherence scores among the models. However, while its BLEU score is the highest, it slightly trails behind GPT-4 in SARI, demonstrating strong but not leading performance in generating entailments that are semantically aligned with the input.

In the MRPC dataset, which tests the model’s ability to generate paraphrases, Mistral-ReadCtrl leads to a readability gap of 1.66, the highest factuality and coherence scores, and the best BLEU and SARI scores. This highlights its exceptional ability to generate paraphrases that not only adhere closely to the desired readability level but also maintain high levels of linguistic and contextual integrity.

Overall, the data from the unseen tasks confirm Mistral-ReadCtrl’s robust generalization capabilities across different types of text generation tasks, demonstrating its potential as a versatile tool in NLP applications where adapting to varied content types and maintaining consistent quality is critical.

4 Human Evaluation

5 Related Work

Early efforts for readability control in natural language generation (NLG) included microplanning steps to tailor the text to match different target reading levelsMoraes etal. (2016); Agrawal and Carpuat (2019); Marchisio etal. (2019).More recent studies, such as those by Luo etal. (2022), have investigated controllable abstractive and extractive approaches for generating summaries tailored for layman and expert audiences from biomedical documents.Concurrently, recent work Pu and Demberg (2023); Rao and Tetreault (2018); Yao and Yu (2021) examined the ability of the language models to adapt its outputs to different target audiences and writing styles, ranging from formal to informal, whereas Imperial (2022) highlighted that GPT2 models struggle with preserving the linguistic complexity of input prompts.Significant developments have also been made in models for Plain Language Summarization (PLS) from scientific papersDevaraj etal. (2021); Goldsack etal. (2023); Guo etal. (2023), focusing on generating simplified texts that retain the original content’s meaning.Unlike these methods, our novel "readability-controlled instruction learning (ReadCtrl)" method distinctly focuses on fine-grained readability control via direct instruction, allowing for precise adaptation of text complexity. This approach ensures that outputs meet specific readability demands, tested across text simplification, paraphrase generation, and semantic entailment generation tasks. Demonstrating its efficacy and versatility, ReadCtrl performs robustly in both ’seen’ and ’unseen’ settings.

Text Simplification aims to enhance the readability of sentences by reducing their linguistic complexity, with various important societal applications, such as increasing accessibility for those with cognitive disabilities and also for patient education, non-native speakers, and children with reading difficultiesMartin etal. (2020); Cao etal. (2020).Various aspects of simplified outputs have been addressed, includingadapting to specific levelsNishihara etal. (2019),incorporating edit operationsKumar etal. (2020); Mallinson etal. (2020),enforcing lexical and syntactic constraintsMartin etal. (2019),applying linguistically motivated syntactic rulesMaddela etal. (2020),and integrating complex span extraction and lay language generationChen etal. (2018); Kwon etal. (2022); Jiang and Xu (2024); Yao etal. (2023)into the simplification process.In contrast to traditional text simplification, which primarily focuses on controlling the extent of paraphrasing, our approaches are designed to produce succinct and informative output for various tasks in both seen and unseen settings, while maintaining different fine-grained levels of desired readability. Our unique contribution extends readability control beyond mere style transfer to a versatile, instruction-based framework that meets diverse user needs.

Finally, our work follows the instruction tuning techniqueZhang etal. (2023a) to help LLMs learn to follow readability-controlled instructions.Traditional supervised fine-tuning (SFT) techniques often struggle with fine-grained readability control, as they depend on manual annotation or synthetic data generation for enriching readability data, followed by model fine-tuning.This method requires considerable financial and time resources, with repeated tasks for each domain-specific application.Alternatively, recent advances in instruction learning offer a more generalized approach, as highlighted in several studiesWei etal. (2021); Wang etal. (2022); Honovich etal. (2022); Zhang etal. (2023b); Tran etal. (2023).Instruction learning operates on the premise that the model already possesses the necessary knowledge and skills to perform the target task but requires instructional data to learn how to follow the instructions effectively.By using a FLAN-style Instruction Fine-Tuning methodWei etal. (2021), we gathered task-specific instructions for ReadCtrl and conducted fine-tuning. Our evaluations, both automatic and human, on seen and unseen tasks, confirm ReadCtrl’s effectiveness, simplifying the adaptation process and broadening application scope with minimal data needs.

6 Conclusion

The ReadCtrl approach significantly enhances the adaptability of LLMs by dynamically adjusting content complexity to suit various readability requirements. Outperforming mainstream models like GPT-4 in evaluations, Mistral-ReadCtrl demonstrates its capability to produce nuanced, high-quality outputs, thereby showing the promise of personalized content generation.

7 Limitations

In this paper, we propose a new instruction-learning approach to enhance the controllability of readability levels. While this adjustment is not specific to any particular language, we conducted all of our experiments and analysis exclusively on English-language summarization datasets.Additionally, due to the resource limitation, our analysis was limited to Text Simplification (ASSET and WikiSmall datasets), Paraphrase Generation (PAWS and MRPC datasets), and Semantic Entailment Generation (SNLI and MultiNLI datasets), reflecting their prevalent application in NLG studies. Consequently, this paper does not extend to exploring style variations in non-English and other relevant tasks and datasets, such as some mentioned text-to-text generation datasets in the tutorial at ACL 2024Dou etal. (2023).Thus, the scope of this study is confined, and the results may not be universally applicable across different linguistic and stylistic contexts.For readability evaluation, studies have shown that readability formulas may not be ideal for medical textZheng and Yu (2017) because short texts (e.g., abbreviations and fragmented texts rather than complete sentences) are common in EHR notes. In future work, we may explore machine-learning-based approachesZheng etal. (2018) for readability in subdomains.Finally, due to resource constraints, we were unable to have actual grade 2, 5, 8, and 11 students provide pairwise preference feedback during our human evaluation. In the future, we plan to collect human evaluation feedback from more appropriate target groups to enhance the reliability of our results further.

8 Ethics Statement

While Mistral-ReadCtrl has demonstrated a high level of readability controllability on several NLG datasets dataset, this does not imply their use as general controllable interactive models (like some chatbot systems). These models should be thoroughly evaluated before being used in different settings to ensure reliability.

References

• Achiam etal. (2023)Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, FlorenciaLeoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, etal. 2023.Gpt-4 technical report.arXiv preprint arXiv:2303.08774.
• Agrawal and Carpuat (2019)Sweta Agrawal and Marine Carpuat. 2019.Controlling text complexity in neural machine translation.arXiv preprint arXiv:1911.00835.
• Alva-Manchego etal. (2020)Fernando Alva-Manchego, Louis Martin, Antoine Bordes, Carolina Scarton, Benoît Sagot, and Lucia Specia. 2020.Asset: A dataset for tuning and evaluation of sentence simplification models with multiple rewriting transformations.arXiv preprint arXiv:2005.00481.
• Anthropic (2024)Anthropic. 2024.The claude 3 model family: Opus, sonnet, haiku.
• Bowman etal. (2015)SamuelR Bowman, Gabor Angeli, Christopher Potts, and ChristopherD Manning. 2015.A large annotated corpus for learning natural language inference.arXiv preprint arXiv:1508.05326.
• Cao etal. (2020)Yixin Cao, Ruihao Shui, Liangming Pan, Min-Yen Kan, Zhiyuan Liu, and Tat-Seng Chua. 2020.Expertise style transfer: A new task towards better communication between experts and laymen.arXiv preprint arXiv:2005.00701.
• Chen etal. (2023)Jin Chen, Zheng Liu, Xunpeng Huang, Chenwang Wu, QiLiu, Gangwei Jiang, Yuanhao Pu, Yuxuan Lei, Xiaolong Chen, Xingmei Wang, Defu Lian, and Enhong Chen. 2023.When large language models meet personalization: Perspectives of challenges and opportunities.ArXiv, abs/2307.16376.
• Chen etal. (2018)Jinying Chen, Emily Druhl, BalajiPolepalli Ramesh, ThomasK Houston, CynthiaA Brandt, DonnaM Zulman, VarshaG Vimalananda, Samir Malkani, and Hong Yu. 2018.A natural language processing system that links medical terms in electronic health record notes to lay definitions: system development using physician reviews.Journal of medical Internet research, 20(1):e26.
• Coleman and Liau (1975)Meri Coleman and TaLin Liau. 1975.A computer readability formula designed for machine scoring.Journal of Applied Psychology, 60(2):283.
• Devaraj etal. (2021)Ashwin Devaraj, ByronC Wallace, IainJ Marshall, and JunyiJessy Li. 2021.Paragraph-level simplification of medical texts.In Proceedings of the conference. Association for Computational Linguistics. North American Chapter. Meeting, volume 2021, page 4972. NIH Public Access.
• Dolan and Brockett (2005)Bill Dolan and Chris Brockett. 2005.Automatically constructing a corpus of sentential paraphrases.In Third international workshop on paraphrasing (IWP2005).
• Dou etal. (2023)Yao Dou, Philippe Laban, Claire Gardent, and Wei Xu. 2023.Automatic and human-ai interactive text generation.arXiv preprint arXiv:2310.03878.
• Goldsack etal. (2022)Tomas Goldsack, Zhihao Zhang, Chenghua Lin, and Carolina Scarton. 2022.Making science simple: Corpora for the lay summarisation of scientific literature.arXiv preprint arXiv:2210.09932.
• Goldsack etal. (2023)Tomas Goldsack, Zhihao Zhang, Chenghua Lin, and Carolina Scarton. 2023.Domain-driven and discourse-guided scientific summarisation.In European Conference on Information Retrieval, pages 361–376. Springer.
• Gunning (1952)Robert Gunning. 1952.The technique of clear writing.(No Title).
• Guo etal. (2023)Yue Guo, Tal August, Gondy Leroy, Trevor Cohen, and LucyLu Wang. 2023.Appls: A meta-evaluation testbed for plain language summarization.arXiv preprint arXiv:2305.14341.
• Guo etal. (2021)Yue Guo, Wei Qiu, Yizhong Wang, and Trevor Cohen. 2021.Automated lay language summarization of biomedical scientific reviews.In Proceedings of the AAAI Conference on Artificial Intelligence, volume35, pages 160–168.
• Honovich etal. (2022)OrHonovich, Thomas Scialom, Omer Levy, and Timo Schick. 2022.Unnatural instructions: Tuning language models with (almost) no human labor.arXiv preprint arXiv:2212.09689.
• Huang etal. (2023)XuHuang, Jianxun Lian, Yuxuan Lei, Jing Yao, Defu Lian, and Xing Xie. 2023.Recommender ai agent: Integrating large language models for interactive recommendations.arXiv preprint arXiv:2308.16505.
• Imperial (2022)JosephMarvin Imperial. 2022.Uniform complexity for text generation.arXiv preprint arXiv:2204.05185.
• Jiang etal. (2023)AlbertQ Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, DevendraSingh Chaplot, Diego delas Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, etal. 2023.Mistral 7b.arXiv preprint arXiv:2310.06825.
• Jiang and Xu (2024)Chao Jiang and Wei Xu. 2024.Medreadme: A systematic study for fine-grained sentence readability in medical domain.arXiv preprint arXiv:2405.02144.
• Kincaid etal. (1975)JPeter Kincaid, RobertP FishburneJr, RichardL Rogers, and BradS Chissom. 1975.Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel.
• Kirk etal. (2024)HannahRose Kirk, Bertie Vidgen, Paul Röttger, and ScottA Hale. 2024.The benefits, risks and bounds of personalizing the alignment of large language models to individuals.Nature Machine Intelligence, pages 1–10.
• Kumar etal. (2020)Dhruv Kumar, Lili Mou, Lukasz Golab, and Olga Vechtomova. 2020.Iterative edit-based unsupervised sentence simplification.arXiv preprint arXiv:2006.09639.
• Kwon etal. (2022)Sunjae Kwon, Zonghai Yao, HarmonS Jordan, DavidA Levy, Brian Corner, and Hong Yu. 2022.Medjex: A medical jargon extraction model with wiki’s hyperlink span and contextualized masked language model score.arXiv preprint arXiv:2210.05875.
• Laban etal. (2022)Philippe Laban, Tobias Schnabel, PaulN Bennett, and MartiA Hearst. 2022.Summac: Re-visiting nli-based models for inconsistency detection in summarization.Transactions of the Association for Computational Linguistics, 10:163–177.
• Lan etal. (2024)Tian Lan, Wenwei Zhang, Chen Xu, Heyan Huang, Dahua Lin, Kai Chen, and Xian-ling Mao. 2024.Criticbench: Evaluating large language models as critic.arXiv preprint arXiv:2402.13764.
• Luo etal. (2022)Zheheng Luo, Qianqian Xie, and Sophia Ananiadou. 2022.Readability controllable biomedical document summarization.arXiv preprint arXiv:2210.04705.
• Maddela etal. (2020)Mounica Maddela, Fernando Alva-Manchego, and Wei Xu. 2020.Controllable text simplification with explicit paraphrasing.arXiv preprint arXiv:2010.11004.
• Mallinson etal. (2020)Jonathan Mallinson, Aliaksei Severyn, Eric Malmi, and Guillermo Garrido. 2020.Felix: Flexible text editing through tagging and insertion.arXiv preprint arXiv:2003.10687.
• Marchisio etal. (2019)Kelly Marchisio, Jialiang Guo, Cheng-I Lai, and Philipp Koehn. 2019.Controlling the reading level of machine translation output.In Proceedings of Machine Translation Summit XVII: Research Track, pages 193–203.
• Martin etal. (2020)Louis Martin, Angela Fan, Éric DeLaClergerie, Antoine Bordes, and Benoît Sagot. 2020.Muss: Multilingual unsupervised sentence simplification by mining paraphrases.arXiv preprint arXiv:2005.00352.
• Martin etal. (2019)Louis Martin, Benoît Sagot, Eric dela Clergerie, and Antoine Bordes. 2019.Controllable sentence simplification.arXiv preprint arXiv:1910.02677.
• Moraes etal. (2016)Priscilla Moraes, KathleenF McCoy, and Sandra Carberry. 2016.Enabling text readability awareness during the micro planning phase of nlg applications.In Proceedings of the 9th International Natural Language Generation conference, pages 121–131.
• Nishihara etal. (2019)Daiki Nishihara, Tomoyuki Kajiwara, and Yuki Arase. 2019.Controllable text simplification with lexical constraint loss.In Proceedings of the 57th annual meeting of the association for computational linguistics: Student research workshop, pages 260–266.
• Papineni etal. (2002)Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.Bleu: a method for automatic evaluation of machine translation.In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
• Pu and Demberg (2023)Dongqi Pu and Vera Demberg. 2023.Chatgpt vs human-authored text: Insights into controllable text summarization and sentence style transfer.arXiv preprint arXiv:2306.07799.
• Qin etal. (2024)Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, and Dong Yu. 2024.Infobench: Evaluating instruction following ability in large language models.arXiv preprint arXiv:2401.03601.
• Rao and Tetreault (2018)Sudha Rao and Joel Tetreault. 2018.Dear sir or madam, may i introduce the gyafc dataset: Corpus, benchmarks and metrics for formality style transfer.arXiv preprint arXiv:1803.06535.
• Ribeiro etal. (2023)LeonardoFR Ribeiro, Mohit Bansal, and Markus Dreyer. 2023.Generating summaries with controllable readability levels.arXiv preprint arXiv:2310.10623.
• Richardson etal. (2023)Chris Richardson, Yao Zhang, Kellen Gillespie, Sudipta Kar, Arshdeep Singh, Zeynab Raeesy, OmarZia Khan, and Abhinav Sethy. 2023.Integrating summarization and retrieval for enhanced personalization via large language models.arXiv preprint arXiv:2310.20081.
• Senter and Smith (1967)RJSenter and EdgarA Smith. 1967.Automated readability index.Technical report, Technical report, DTIC document.
• Shanahan etal. (2023)Murray Shanahan, Kyle McDonell, and Laria Reynolds. 2023.Role play with large language models.Nature, pages 1–6.
• Sun etal. (2023)Jiao Sun, Yufei Tian, Wangchunshu Zhou, Nan Xu, Qian Hu, Rahul Gupta, JohnFrederick Wieting, Nanyun Peng, and Xuezhe Ma. 2023.Evaluating large language models on controlled generation tasks.arXiv preprint arXiv:2310.14542.
• Tran etal. (2023)Hieu Tran, Zhichao Yang, Zonghai Yao, and Hong Yu. 2023.Bioinstruct: Instruction tuning of large language models for biomedical natural language processing.arXiv preprint arXiv:2310.19975.
• Vajjala (2021)Sowmya Vajjala. 2021.Trends, limitations and open challenges in automatic readability assessment research.arXiv preprint arXiv:2105.00973.
• Wang etal. (2023)Peiyi Wang, Lei Li, Liang Chen, Dawei Zhu, Binghuai Lin, Yunbo Cao, QiLiu, Tianyu Liu, and Zhifang Sui. 2023.Large language models are not fair evaluators.arXiv preprint arXiv:2305.17926.
• Wang etal. (2022)Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, NoahA Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022.Self-instruct: Aligning language models with self-generated instructions.arXiv preprint arXiv:2212.10560.
• Wei etal. (2021)Jason Wei, Maarten Bosma, VincentY Zhao, Kelvin Guu, AdamsWei Yu, Brian Lester, Nan Du, AndrewM Dai, and QuocV Le. 2021.Finetuned language models are zero-shot learners.arXiv preprint arXiv:2109.01652.
• Williams etal. (2017)Adina Williams, Nikita Nangia, and SamuelR Bowman. 2017.A broad-coverage challenge corpus for sentence understanding through inference.arXiv preprint arXiv:1704.05426.
• Xu etal. (2016)Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. 2016.Optimizing statistical machine translation for text simplification.Transactions of the Association for Computational Linguistics, 4:401–415.
• Yao etal. (2023)Zonghai Yao, NandyalaSiddharth Kantu, Guanghao Wei, Hieu Tran, Zhangqi Duan, Sunjae Kwon, Zhichao Yang, Hong Yu, etal. 2023.Readme: Bridging medical jargon and lay understanding for patient education through data-centric nlp.arXiv preprint arXiv:2312.15561.
• Yao and Yu (2021)Zonghai Yao and Hong Yu. 2021.Improving formality style transfer with context-aware rule injection.arXiv preprint arXiv:2106.00210.
• Zeng etal. (2023)Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, YuMeng, Tanya Goyal, and Danqi Chen. 2023.Evaluating large language models at evaluating instruction following.arXiv preprint arXiv:2310.07641.
• Zhang etal. (2023a)Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, etal. 2023a.Instruction tuning for large language models: A survey.arXiv preprint arXiv:2308.10792.
• Zhang etal. (2023b)Xinlu Zhang, Chenxin Tian, Xianjun Yang, Lichang Chen, Zekun Li, and LindaRuth Petzold. 2023b.Alpacare: Instruction-tuned large language models for medical application.arXiv preprint arXiv:2310.14558.
• Zhang etal. (2019)Yuan Zhang, Jason Baldridge, and Luheng He. 2019.Paws: Paraphrase adversaries from word scrambling.arXiv preprint arXiv:1904.01130.
• Zheng and Yu (2017)Jiaping Zheng and Hong Yu. 2017.Readability formulas and user perceptions of electronic health records difficulty: a corpus study.Journal of medical Internet research, 19(3):e59.
• Zheng etal. (2018)Jiaping Zheng, Hong Yu, etal. 2018.Assessing the readability of medical documents: a ranking approach.JMIR medical informatics, 6(1):e8611.
• Zheng etal. (2024)Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, ZiLin, Zhuohan Li, Dacheng Li, Eric Xing, etal. 2024.Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36.
• Zhong etal. (2022)Ming Zhong, Yang Liu, DaYin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. 2022.Towards a unified multi-dimensional evaluator for text generation.arXiv preprint arXiv:2210.07197.
• Zhou etal. (2023)Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, YiLuan, Denny Zhou, and LeHou. 2023.Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911.
• Zhu etal. (2010)Zhemin Zhu, Delphine Bernhard, and Iryna Gurevych. 2010.A monolingual tree-based translation model for sentence simplification.In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1353–1361.

Appendix A Readability Metrics

Appendix B More details about human evaluation

We provide additional details on our human evaluation setup. Human preference annotators are 5 students who have completed a bachelor’s degree or above from an American university and are fluent in English.We add some tasks with known answers (i.e., cases where the most/least readable and good/bad quality text should be clear), enabling us to estimate the accuracy of annotators who work on these. Annotators with low accuracy on tasks with known answers are automatically removed from our worker pool. Only the annotators who passed these final tests were accepted to work on the human preference in this paper.We gave annotators fair compensation (20$/hrs).

Appendix C More details about LLM evaluation

Appendix D Hyper-parameter Settings

The experiments were executed using the version 4.37.1 of the transformers library released by Hugging Face. In Table 3, we report the hyperparameters used to train the models on our combined dataset. We use the Adam optimizer and employ a linearly decreasing learning rate schedule with warm-up step is 200. In this section, we detail our experimental setup, the datasets employed, and the evaluation strategy adopted for assessing the performance of our instruction-tuned LLMs in various BioNLP tasks. Furthermore, all experiments were conducted using two Nvidia A100 GPUs, each with 40 GB of memory. The CPU used was an Intel Xeon Gold 6230 processor, and the system was equipped with 192 GB of RAM.

Appendix E Experiments with GPT3.5, GPT4, Claude-3

All of our experiments were conducted on the version of GPT3.5, GPT4 and Claude 3 between 25 March 2023 and 13 April 2024 by using the OpenAI’s API.10 We set temperature = 1, top_p=1, frequency penalty = 0, and presence penalty = 0.

Appendix F ReadCtrl instruction following evaluation setting

We have plotted Figures 1, 6, 7, 8, 9, 10, and 11 by calculating the readability scores or reading levels of the outputs generated in response to prompts that request specific reading levels ranging from 1 to 12. These calculations were performed across all test sets of the six datasets mentioned in the Experiment section. Additionally, we calculated the standard deviation of the readability scores across these test sets to assess the consistency of the output’s readability.

Appendix G Examples of output generated by Mistral-ReadCtrl and GPT4 during ReadCtrl instruction following evaluation

Tables4 present distinct levels of output generated by the Mistral-ReadCtrl and GPT4 and their readability scores given by Flesch-Kincaid Grade Level (FKG), Gunning fog index (GFI), and Coleman-Liau index (CLI) metrics.

We will delve into the observed discrepancies between the Readability Gap and the performance curves in our evaluation, as demonstrated by our results for the PAWS and MultiNLI datasets. The Readability Gap, calculated as the average difference between the actual readability score of the output and the requested readability score across all samples, shows intriguing variations in behavior across different datasets.

For the PAWS dataset, although the Readability Gap appears almost perfect in Table 1, the corresponding curve does not exhibit as favorable performance. This anomaly can be attributed to the output readability distribution of PAWS, which is somewhat concentrated within a specific range (typically between 4-8). While this concentration allows for excellent performance within this median range, it leads to a less generalized performance across the full spectrum of readability levels (from 1-12). Therefore, even a small Readability Gap in numerical terms may not accurately reflect an evenly distributed ability to target all requested readability levels.

Conversely, the MultiNLI dataset exhibits a larger Readability Gap in Table 1, yet the performance curve approaches perfection. This suggests that while the average gap is larger, the outputs are more uniformly distributed across the entire range of readability levels, allowing for closer adherence to the target levels across a broader spectrum. This indicates a more generalized and adaptable performance despite the numerically larger gap.

This analysis underscores the importance of considering both the Readability Gap and the distribution of output readability scores when assessing model performance. A low Readability Gap might suggest excellent average performance but could conceal poor adaptability across a range of readability levels. Conversely, a higher Readability Gap might indicate a more uniform distribution of performance across all levels, suggesting a different kind of effectiveness.

Further investigation into these patterns for all six datasets employed in our study reveals similar trends.

Appendix H Prompts

Table 4: All prompts used in our work.