初始化项目，由ModelHub XC社区提供模型

Model: roger33303/Jarvis_granite_4.1_3b_finetuned_16bit Source: Original Platform
2026-07-08 08:30:12 +08:00
commit 2d9ab25841
18 changed files with 603008 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,306 @@
+---
+base_model: unsloth/granite-4.1-3b
+tags:
+- text-generation-inference
+- transformers
+- unsloth
+- granite
+- jarvis
+- persona
+- fine-tuned
+- qlora
+- ddp
+license: apache-2.0
+language:
+- en
+datasets:
+- roger33303/jarvis_deep_dataset
+metrics:
+- bertscore
+- bleu
+- rouge
+model-index:
+- name: Jarvis_granite_4.1_3b_finetuned_16bit
+  results:
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: roger33303/jarvis_deep_dataset
+      type: roger33303/jarvis_deep_dataset
+      split: validation
+      args:
+        num_few_shot: 0
+    metrics:
+    - type: bertscore
+      value: 0.8641
+      name: BERTScore F1
+      verified: false
+    - type: bleu
+      value: 0.1555
+      name: BLEU
+      verified: false
+    - type: rouge
+      value: 0.2408
+      name: ROUGE-L
+      verified: false
+---
+
+# Jarvis - Granite 4.1 3B Fine-tuned (16bit)
+
+A JARVIS-persona fine-tune of IBM's [Granite 4.1 3B](https://huggingface.co/ibm-granite/granite-4.1-3b-instruct), trained to respond in the dry, formal, and subtly witty style of J.A.R.V.I.S. from the Iron Man universe.
+
+> "I am fully operational and, as ever, at your disposal, sir."
+
+## Evaluation Results
+
+Evaluated on the full 6,983-row validation split using the HuggingFace `evaluate` library. Inference was run using vLLM (tensor_parallel_size=2) on 2x NVIDIA T4 GPUs.
+
+To properly measure the lift from fine-tuning, the base model (`unsloth/granite-4.1-3b`, no fine-tuning) was also evaluated on the identical validation set, prompted with the same JARVIS system prompt used to generate the training data. This isolates what fine-tuning actually contributed versus what prompting alone can achieve.
+
+| Metric | Base + system prompt | Fine-tuned 16bit | Notes |
+|---|---|---|---|
+| BERTScore F1 | 0.8420 | 0.8641 | Primary metric: semantic and stylistic alignment |
+| BLEU | 0.0806 | 0.1555 | Lexical overlap with reference outputs |
+| ROUGE-L | 0.1803 | 0.2408 | Longest common subsequence recall |
+| Persona adherence delta | -0.1174 | +0.0093 | Model vs. reference persona score (0 is ideal) |
+| Length ratio | 2.390 | 1.232 | Model output length divided by reference length (1.0 is ideal) |
+| Self-BLEU | 0.1017 | 0.0584 | Lower indicates healthier diversity, less templated output |
+
+Why BERTScore is the primary metric: for persona fine-tuning, lexical overlap (BLEU/ROUGE) is a poor proxy for style fidelity. BERTScore captures semantic and contextual alignment at the embedding level, making it far more sensitive to whether the model has genuinely adopted the JARVIS voice rather than just copying words.
+
+Interpretation:
+
+- Persona delta is the clearest signal of fine-tuning's impact. The base model, even when given the exact JARVIS system prompt, undershoots the target persona by -0.117, meaning it inconsistently uses "sir," contractions, and JARVIS-style phrasing. Fine-tuning flips this to +0.009, essentially matching the target persona intensity.
+- Length ratio tells a similar story. The base model is 2.39 times more verbose than the reference responses, rambling without fine-tuned guidance on appropriate response length. Fine-tuning brings this down to 1.232, much closer to the target.
+- BERTScore F1 improved by 0.022, confirming genuine semantic and stylistic alignment gains, not just surface-level word overlap.
+- Self-BLEU dropped from 0.1017 to 0.0584, meaning fine-tuned outputs are less repetitive and templated than the base model's. The base model leans on generic catchphrases when prompted into a persona it has not learned.
+- A BERTScore F1 of 0.864 at only approximately 35% training completion signals strong style transfer with room to improve further on a full training run.
+
+### Base vs. fine-tuned 16bit, all metrics
+
+<img src="eval_imgs/jarvis_combined_base_vs_16bit.png" alt="BLEU, ROUGE-L, BERTScore F1, Self-BLEU comparison between base and fine-tuned 16bit model" width="480"/>
+
+### Length ratio, base vs. fine-tuned 16bit
+
+<img src="eval_imgs/jarvis_length_ratio_base_vs_16bit.png" alt="Length ratio comparison between base and fine-tuned 16bit model" width="480"/>
+
+### Persona delta, base vs. fine-tuned 16bit
+
+<img src="eval_imgs/jarvis_persona_delta_base_vs_16bit.png" alt="Persona delta comparison between base and fine-tuned 16bit model" width="480"/>
+
+## The JARVIS Persona
+
+The model was fine-tuned to follow a 10-rule JARVIS dialect style guide. Every training example was verified against these rules.
+
+1. Always addresses the user as "sir," with no exceptions
+2. No contractions: "I am" not "I'm," "do not" not "don't," "cannot" not "can't"
+3. Formal vocabulary substitutions: "ascertain" over "find out," "endeavour" over "try," "assist" over "help"
+4. Six dry wit archetypes:
+   - Understatement: describing catastrophic events with mild detachment
+   - Deadpan observation: noting obvious things with academic precision
+   - Polite sarcasm: technically correct but pointed
+   - Reluctant compliance: following absurd instructions with dignified resignation
+   - Ironic precision: pedantic accuracy that highlights absurdity
+   - Understated concern: alarm expressed through studied calm
+5. Response length matching: short inputs get short outputs; complex queries get proportionate depth
+6. Sentence structure preferences: subordinate clauses, measured pacing, no bullet-point lists for conversational responses
+7. Banned casual words: "okay," "sure," "awesome," "great," "totally," "literally," "basically," "cool," "wow," "yeah," and similar
+8. Out-of-character handling: breaks persona gracefully when genuinely unable to help, without breaking immersion unnecessarily
+9. Tone calibration by situation: crisis is focused and efficient, casual engages dry wit, technical is precise and formal
+10. Complete examples: every rule illustrated with before and after examples in the training data generation prompt
+
+Example exchange:
+
+```
+User: What is the weather like today?
+
+JARVIS: I am afraid I do not have access to your current location's
+meteorological data at this moment, sir. You may wish to consult a
+dedicated weather service, or if you provide your coordinates I can
+make the appropriate enquiry on your behalf.
+```
+
+## Model Details
+
+| Property | Value |
+|---|---|
+| Base model | [unsloth/granite-4.1-3b](https://huggingface.co/unsloth/granite-4.1-3b) |
+| Architecture | Granite 4.1, 3.40B parameters, 40 layers |
+| Fine-tuning method | QLoRA (4-bit quantization during training) |
+| LoRA rank | r=16, alpha=32 |
+| LoRA target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, shared_mlp.input_linear, shared_mlp.output_linear |
+| Max sequence length | 1536 tokens |
+| Training steps completed | 3,000 of 8,476 (approximately 35% of full run) |
+| Saved as | bfloat16 merged weights (full model, no adapter required) |
+| License | Apache 2.0 |
+| Developed by | [roger33303](https://huggingface.co/roger33303) |
+
+## Training Dataset
+
+Fine-tuned on [roger33303/jarvis_deep_dataset](https://huggingface.co/datasets/roger33303/jarvis_deep_dataset), a 69,836-row instruction-response dataset constructed by rewriting [WizardLMTeam/WizardLM_evol_instruct_70k](https://huggingface.co/datasets/WizardLMTeam/WizardLM_evol_instruct_70k) into the JARVIS persona using the DeepSeek API with 400 concurrent async workers and fault-tolerant resume logic.
+
+| Split | Rows |
+|---|---|
+| Train | 60,853 |
+| Validation | 6,983 |
+| Test | 2,000 |
+| Total | 69,836 |
+
+Splits were constructed using quantile-based stratified sampling on token length (n_bins=3, boundaries via `np.percentile`) to ensure balanced representation of short, medium, and long instructions across all splits. No hardcoded length boundaries; fully data-driven.
+
+## Training Configuration
+
+Training was performed using [Unsloth](https://github.com/unslothai/unsloth) and HuggingFace TRL with Accelerate DDP across 2x NVIDIA T4 GPUs on Kaggle.
+
+```python
+SFTConfig(
+    per_device_train_batch_size = 1,       # DDP stability (Unsloth known issue with batch>1)
+    gradient_accumulation_steps = 4,       # effective batch = 1 x 4 x 2 GPUs = 8
+    num_train_epochs            = 1,
+    learning_rate               = 1e-4,
+    optim                       = "adamw_8bit",
+    lr_scheduler_type           = "cosine",
+    warmup_steps                = 100,
+    weight_decay                = 0.01,
+    eval_strategy               = "steps",
+    eval_steps                  = 500,     # roughly 17 evals across the run
+    save_steps                  = 250,
+    max_seq_length              = 1536,
+    ddp_find_unused_parameters  = False,   # required for DDP + LoRA
+    mixed_precision             = "bf16",
+)
+```
+
+DDP setup (Accelerate config):
+
+```yaml
+compute_environment: LOCAL_MACHINE
+distributed_type: MULTI_GPU
+num_processes: 2
+num_machines: 1
+mixed_precision: bf16
+use_cpu: false
+```
+
+Key DDP requirements:
+
+- `device_map` must not be passed to `FastLanguageModel.from_pretrained`, as it conflicts with DDP process placement
+- All logging, hub pushes, and checkpoint saves gated to `PartialState().is_main_process`
+- `ddp_find_unused_parameters=False` required for LoRA and DDP to avoid hanging
+
+Throughput: approximately 4.34 s/it (2x T4 DDP) versus approximately 7.9 s/it (single T4), roughly a 1.8x speedup.
+
+Training on responses only: used `train_on_responses_only()` with Granite's role tokens.
+
+```python
+instruction_part = "<|start_of_role|>user<|end_of_role|>"
+response_part    = "<|start_of_role|>assistant<|end_of_role|>"
+```
+
+Checkpoint resume: automatically detects latest checkpoint in `output_dir` and resumes, with no manual configuration needed between Kaggle sessions.
+
+## Usage
+
+### With Transformers
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+model_id = "roger33303/Jarvis_granite_4.1_3b_finetuned_16bit"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+
+messages = [{"role": "user", "content": "Explain quantum entanglement."}]
+
+prompt = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=512,
+    temperature=0.7,
+    do_sample=True,
+)
+
+response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
+print(response)
+```
+
+### With vLLM (recommended for multi-GPU inference)
+
+```python
+from vllm import LLM, SamplingParams
+from transformers import AutoTokenizer
+
+tokenizer = AutoTokenizer.from_pretrained("roger33303/Jarvis_granite_4.1_3b_finetuned_16bit")
+
+llm = LLM(
+    model="roger33303/Jarvis_granite_4.1_3b_finetuned_16bit",
+    tensor_parallel_size=2,
+    max_model_len=1536,
+    gpu_memory_utilization=0.85,
+    dtype="float16",   # T4 does not support bfloat16
+)
+
+sampling_params = SamplingParams(temperature=0.7, max_tokens=512)
+
+messages = [{"role": "user", "content": "What is the status of the reactor?"}]
+prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+
+outputs = llm.generate([prompt], sampling_params)
+print(outputs[0].outputs[0].text.strip())
+```
+
+## Reproducibility
+
+All code used to train and evaluate this model is publicly available.
+
+| Resource | Link |
+|---|---|
+| Training notebook (with outputs) | [Kaggle: parallel-training-on-multiple-gpus](https://www.kaggle.com/code/abhinavatall/parallel-training-on-multiple-gpus?scriptVersionId=328372335) |
+| Evaluation notebook (with outputs) | [Kaggle: vllm-evaluate-16-bit-model-evaluation](https://www.kaggle.com/code/abhinavatall/vllm-evaluate-16-bit-model-evaluation) |
+| Training dataset | [roger33303/jarvis_deep_dataset](https://huggingface.co/datasets/roger33303/jarvis_deep_dataset) |
+
+## Evaluation Pipeline
+
+The evaluation pipeline used vLLM for fast batched inference (batch_size=64) with checkpoint-resume support, then computed all metrics using the HuggingFace `evaluate` library.
+
+Persona adherence score is a composite metric:
+
+```python
+score = (
+    0.35 * has_sir          +   # addresses user as "sir"
+    0.30 * no_contractions  +   # avoids contractions
+    0.20 * no_banned_words  +   # avoids casual vocabulary
+    0.15 * has_preferred_phrase # uses formal JARVIS phrases
+)
+```
+
+The persona delta (model score minus reference score) of +0.009 indicates the model matches the target persona intensity, neither undershooting nor overshooting into caricature.
+
+The base model baseline was generated using the identical JARVIS system prompt (from the dataset-creation pipeline's `prompt_lib.py`) paired directly with each validation instruction, with no fine-tuning and no rewrite wrapper, to test whether prompting alone can achieve the persona that fine-tuning was designed to teach.
+
+## Limitations
+
+- Training was stopped at step 3,000 of 8,476 (approximately 35%) due to Kaggle session timeout. A full training run is expected to further improve all metrics.
+- The model was trained on instruction-following data rewritten into the JARVIS persona. It may occasionally break character on highly technical or ambiguous queries.
+- Outputs tend to be approximately 23% longer than reference responses. This is consistent with JARVIS's verbosity but may be reduced with additional training.
+- The base model (Granite 4.1 3B) has a knowledge cutoff; the fine-tune does not extend factual knowledge.
+
+## Acknowledgements
+
+Trained with [Unsloth](https://github.com/unslothai/unsloth) for memory-efficient QLoRA fine-tuning on Kaggle's free GPU tier.
+
+[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="150"/>](https://github.com/unslothai/unsloth)