CuBERT: BERT Pretrained on Programming Languages

CuBERT: BERT Pretrained on Programming Languages

Category: Natural Language Processing
License: Apache-2.0
Model Type: Generative AI
CuBERT is a collection of BERT-based models trained on source code, specifically five popular programming languages: Python, Java, JavaScript, PHP, and Ruby. Developed by Google Research, CuBERT leverages the BERT architecture to understand and process source code for a variety of downstream software engineering tasks. It is designed to support classification and sequence prediction tasks on code, enabling powerful static analysis and code intelligence capabilities.

Key Features

  • CuBERT is a collection of BERT-based models trained on source code, specifically five popular programming languages: Python, Java, JavaScript, PHP, and Ruby. Developed by Google Research, CuBERT leverages the BERT architecture to understand and process source code for a variety of downstream software engineering tasks. It is designed to support classification and sequence prediction tasks on code, enabling powerful static analysis and code intelligence capabilities.
GitHub Demo Video Youtube

Similar Projects

PG-19: A Dataset for Long-Form Language Modeling

PG-19: A Dataset for Long-Form Language Modeling

Natural Language Processing
llmtranslate

llmtranslate

Natural Language Processing
GPT‑4o Language Translator

GPT‑4o Language Translator

Natural Language Processing
Transformers by Hugging Face

Transformers by Hugging Face

Natural Language Processing
GPT-Neo: Open Source GPT-3 Style Language Model with TPU & GPU Support

GPT-Neo: Open Source GPT-3 Style Language Model with TPU & GPU Support

Natural Language Processing
XLNet: Generalized Autoregressive Pretraining for Language Understanding

XLNet: Generalized Autoregressive Pretraining for Language Understanding

Natural Language Processing