Vortex Language Model (VLM) Documentation

Overview

VLM stands for Vortex Language Model, a series of transformer-based models developed by Vortex Intelligence. The models are designed for tasks such as text generation, reasoning, and instruction following. Each version of VLM is structured in three training stages for progressive refinement.

Note on Hardware: Training is performed on different GPU configurations depending on availability - sometimes using 1× NVIDIA RTX A5000, sometimes 1× A40, and occasionally 1× RTX 5060 Ti with 16GB VRAM.

Model Structure

Each VLM version follows a three-stage pipeline:

K1: Trained from scratch (base model)
K2: Fine-tuned on broader/general-purpose data
K3: Fine-tuned for clarity and simplicity

K stands for Knowledge, with higher numbers representing more advanced training stages. Higher doesn't mean the model has more parameters!

Versions and Training Details

VLM 1

Parameters: 124M
Training Time: ~4 hours per stage
Final Loss (all stages): ~3.0
K1: Trained on tatsu-lab/alpaca and a small custom dataset
K2: Fine-tuned K1 on Elriggs/openwebtext-100k
K3: Fine-tuned K2 on rahular/simple-wikipedia

VLM 1.1

Parameters: 355M
Training Time: ~4 hours per stage
Target Final Loss: ~1.0
K1: Currently training on ------ and ------

Contact & More

Developed and maintained by Vortex Intelligence.