Product Feedback & Architecture Proposal

I came across Nugen while applying for it, and I enjoyed exploring the platform. Thank you for the $50 in credits. Here are a few suggestions and improvements I would like to see in Nugen.

Issues

  1. Model is not registered when an alignment is created without benchmark_idWhen we create an alignment via the API without passing a benchmark ID, the alignment reaches 100% and READY, but no model appears in /models/aligned.
  2. Domain alignment cannot be started from the UIRecent changes removed the built-in models that support domain alignment. Users must import a compatible model first, but the UI has no model-import flow, so alignment cannot be done from the UI.
  3. Benchmark questions and answers are not shown in the UI

Feature suggestions

  1. Accept multiple benchmark IDs when creating an alignment project
  2. Add APIs for checkpoints, continued training, and retraining
  3. Add a PATCH API for editing alignment project details
  4. Allow previously aligned models as a training base (currently this requires exporting to Hugging Face and importing the model, which is a long process)
  5. Expose epochs and training hyperparameters
  6. Add base model selection and model import to the UI

Current model

[Alignment Project] ──(1:1)──► [Training Run] ──(only if benchmark exists)──► [Model Artifact]

Problem: Adding documents or rerunning training requires a new project, which creates v1, v2, and v3 clutter. Training cannot be resumed, and checkpoints cannot be selected.

Suggested structure

  1. ProjectEditable configuration containing documents, base model, and settings.
  2. RunImmutable training execution created by the project.
  3. CheckpointWeights and metrics produced by a run.
  4. ModelStable model ID and endpoint pointing to related checkpoint.
  5. EvaluationIndependent test of any checkpoint against any benchmark.

Proposed workflow

PROJECT ├── RUN 1 (initial training) ── checkpoint 1.1 ├── RUN 2 (retrain) ── checkpoint 2.1 └── RUN 3 (retrain) ── checkpoint 3.1 └── RUN 4 (continued from 3) ── checkpoint 3.2

The main idea is to separate the recipe (base model + docs + benchmark) and the execution (training run).

API changes

  1. Add project editingPATCH /alignment-projects/{id} updates the recipe without creating another top-level project.
  2. Make runs explicitPOST /alignment-projects/{id}/runs supports clean retraining and an explicit warm_start_from run or checkpoint.
  3. Expose checkpointsList checkpoints, pick, rollback, evaluate, and deploy.
  4. Separate evaluationsPOST /evaluations accepts a candidate run or checkpoint and one or more reusable benchmarks.