CTRL is a 1.63 billion-parameter causal (unidirectional) Transformer model trained to generate text conditioned on control codes that govern style, content, and task-specific behavior. These control codes, derived from naturally co-occurring structures in raw text, allow for explicit control over text generation, enabling users to influence aspects such as domain, style, topics, dates, entities, relationships between entities, plot points, and task-related behavior. The model was trained on a large corpus of approximately 140 GB of text data, with the first token reserved as a control code.
Key Features
Conditional Text Generation: Generates text conditioned on control codes specifying various aspects like domain, style, and content.
Large-Scale Pretraining: Trained on a vast corpus of approximately 140 GB of text data.
Control Code Mechanism: Utilizes control codes to guide the generation process, allowing for fine-grained control over the output.
Versatility: Applicable to a wide range of natural language processing tasks, including creative writing, summarization, and dialogue generation.
Open Source: Released under the BSD 3-Clause License, promoting transparency and reproducibility in research.
Android DevHub
.png)
.png)
.png)
.png)
.png)