Reprise du message précédent :

Windows ou linux ?

 
 
Windows 11 Professionnal.

Vous avez un vieux PC compatible AVX2 avec 8GO de RAM (j'ai bien dit RAM. pas VRAM, pas d'accélérateur dédié)  ?  
 
C'est Parfait pour héberger Kimi K3 : https://github.com/FareedKhan-dev/kimi-k3-in-c  
 

 
Ca a reussi avec le guide suivant:
 
# Enable PCIe P2P on pc3090 (dual 3090-class, vLLM TP=2)
 
## Context
 
Both GPUs run without peer access — `nvidia-smi topo -p2p rwnap` reports `GNS`, the stock
driver's software block on GeForce cards. Every tensor-parallel all-reduce in the vLLM TP=2
path is therefore staged through host memory instead of crossing PCIe directly.
 
The estate's own cross-rig data (`/opt/ai/club-3090/docs/PCIE_P2P.md` §6) puts the win for
exactly this configuration — dual 3090, TP=2, vLLM — at **prefill +12–14%, TTFT −13–15%,
decode +7–12%**, with the spec-decode path (this slug runs the DFlash2 drafter) at +19–22%
in #95. On a 90K prefill currently taking ~70 s that is roughly 8 s per long prompt. §6 is
equally clear this buys nothing on llama.cpp/GGUF — the case rests entirely on the vLLM TP path.
 
**This was attempted on 2026-08-13 and abandoned within an hour.** Timestamps reconstruct it:
baseline bench 09:47, modules built 10:06, `bench-p2p.txt` created **empty** at 10:27, revert
confirmed 10:44. An empty bench file means the harness got nothing — the server never came up.
Had the stock module silently kept loading, the bench would have produced a full,
baseline-identical file instead. So the likely failure was the §8 *hollow grant*: the patched
module loaded, granted P2P, `detect_nvlink.sh` auto-enabled the P2P path, and NCCL hung at
`pynccl` init — **with the IOMMU translating**, which §8 lists as the first cause to check.
 
The IOMMU is still translating: `amd_iommu=on` is rejected by this kernel
(`AMD-Vi: Unknown option - 'on'`), leaving `Default domain type: Translated`. That one-line fix
is the most likely reason this attempt succeeds where August did not.
 
**The version situation forces a decision, and it is urgent on its own terms.** The installed
610.43.02 has been superseded: its .deb is **gone from the pool** (verified — `apt-get download`
404s), while `610.57.04-0ubuntu0.26.04.3` is live in both `resolute-updates` and
`resolute-security` and fetchable today. `50unattended-upgrades` allows
`${distro_codename}-security`, nothing is held, and the local apt index is 11 days stale — so
the driver **will** upgrade itself once the index refreshes. A kernel bump is queued too
(`linux-image-generic` 7.0.0-31.31 vs 7.0.0-30.30 installed). Building a custom module against
a version that no longer exists, on a box poised to upgrade underneath it, is the wrong
foundation. So this plan takes both upgrades deliberately, first.
 
**Intended outcome:** P2P engaged and *proven* to carry correct data, vLLM serving on it with a
measured same-sitting A/B, a driver that survives kernel upgrades by itself, and a rollback that
does not depend on the network.
 
## Target state
 
| | from | to |
|---|---|---|
| kernel | 7.0.0-30-generic | 7.0.0-31-generic |
| driver | 610.43.02 (deb gone from pool) | 610.57.04-0ubuntu0.26.04.3 |
| kernel module | stock `nvidia-dkms-610-open` | `nvidia/610.57.04-p2p` via DKMS, from fork branch `610.57.04-p2p-v3` |
| IOMMU | translating | `iommu=pt` |
| `NVLINK_MODE` | unset (→ `auto`) | `pcie_p2p` |
 
**3 reboots.** Each phase changes one variable; serving stays pinned to the pre-patch path
until Phase 6, so the driver swap is proven before anything serving depends on it.
 
## Execution status (updated 2026-09-14 22:40)
 
| phase | state |
|---|---|
| 1 IOMMU passthrough | ✅ `iommu=pt`, `Default domain type: Passthrough`, no `Unknown option` |
| 2 kernel + driver upgrade | ✅ `7.0.0-31-generic` / `610.57.04`, stock DKMS rebuilt for -29/-30/-31, all four holds applied |
| 3 rollback re-prime | ✅ 610.57.04 debs + `-31` module tarball + 610.57.04 source cached; 610.43.02 artifacts retained |
| 3 baseline | ⚠️ **partial** — see below |
| 4 build `610.57.04-p2p-v3` | ✅ passed, see below |
| 5–8 | not started |
 
**Baselines captured (stock 610.57.04, P2P off):**
- `p2p-validate.sh` → **exit 0, HEALTHY**, both arms correct
- bandwidth → **peer 5.76 GB/s vs host-staged 6.65 GB/s (0.87×)**, "no peer advantage" — 6.65 GB/s
  is the number P2P must beat
- `p2p-check.sh` → correctness PASS, 5.8 GB/s
 
**Still outstanding in Phase 3** — both ran before the server finished booting and must be redone
against the live endpoint (`localhost:8104`, *not* 8010 — 8010 was the old qwen3.6 container):
- `verify-full.sh` → 8 checks failed on an unreachable server
- `bench.sh` → output file is 242 bytes of "service not reachable"; it would poison the Phase 8
  A/B if left in place
 
**Gate 4 result — passed clean:**
- all five modules `version 610.57.04` (= running driver), `vermagic 7.0.0-31-generic` (= running
  kernel); licenses `Dual MIT/GPL`, `nvidia-peermem` `Dual BSD/GPL` (matches stock)
- build log 15749 lines, **0 errors**, no modpost failures, no undefined symbols
- patch confirmed in the worktree: 31 files / 1185 insertions vs upstream `e4a5faa2`, core commit
  `94b69ebc`, `ENABLE_RESIZABLE_BAR` default = 1. Much larger than the 610.43.02 patch (8 files /
  83 insertions) — it touches `p2p_caps.c`, `kern_bus_*` across maxwell/pascal/turing,
  `nv_gpu_ops.c`, `mem_desc.c`, `io_vaspace.c`
- `patch-libcuda-p2p.py` is a **separate userspace step, not needed here** — mixed-generation only,
  and this pair is both GA102
- the branch also carries an auto-enabled experimental hugepage `cudaHostRegister` fast path;
  **dormant on this rig** (HugePages_Total 0, no hugetlbfs bind). It would go live if
  `scripts/hugepages.sh` were ever used — remember this if odd behaviour appears later
 
## Preconditions — verified, no action needed
 
| item | state |
|---|---|
| Resizable BAR / Above 4G | **on** — BAR1 32768 MiB per card, ≥ VRAM (the v3 README's hard prerequisite) |
| Topology | GPU0 `01:00.0` on root port `00:01.1`, GPU1 `03:00.0` on `00:01.3`, both Gen4 x8, `PHB` |
| GPU pair | both GA102 — **same generation**, so per the v3 README no `libcuda` patch is needed |
| NVLink | not possible — the 3090 Ti has no NVLink fingers. BAR1 path only |
| Secure Boot | disabled — no module signing |
| initramfs | contains **no** nvidia modules — no `update-initramfs` step |
| Recovery | headless, no display manager, sshd active — a driver that fails to load cannot lock you out |
 
## Locked decisions
 
1. **Upgrade kernel + driver first**, deliberately, then patch `610.57.04-p2p-v3`.
2. **Install via DKMS** as `nvidia/610.57.04-p2p` — auto-rebuilds on kernel upgrades and lands
   in `updates/dkms`, where depmod's `search updates ubuntu built-in` order makes it win.
3. **Ship the default custom all-reduce config** (`NVLINK_MODE=pcie_p2p`), gated hard on
   `verify-full.sh`.
4. **Full same-sitting A/B**, baselined on the *final* stock stack.
 
## Prep required before starting (the cached artifacts are version-pinned)
 
These were written for 610.43.02 and must be re-pointed at 610.57.04 — a mechanical edit:
 
- `~/p2p-run/dkms.conf` — `PACKAGE_VERSION`
- `~/p2p-run/install-dkms.sh` — `STOCK_VER`, `P2P_VER`, `DST`
- `~/p2p-run/rollback.sh` — `STOCK_VER`, `P2P_VER`
- `~/p2p-run/rollback-cache/` — re-primed in Phase 3 against the new stack. Keep the existing
  610.43.02 tarballs as a deeper fallback; they are the only copy of that version left.
 
## Files this plan modifies
 
- `/etc/default/grub` — `amd_iommu=on` → `iommu=pt` (line 10, leaves `iomem=relaxed` intact)
- `/opt/ai/club-3090/.env` — add `NVLINK_MODE`, `force_off` then `pcie_p2p`
- `/usr/src/nvidia-610.57.04-p2p/` — new, staged by `install-dkms.sh`
- DKMS registry — stock `nvidia/610.57.04` removed, `nvidia/610.57.04-p2p` added
- apt holds on the nvidia packages, applied **after** the upgrade
 
No source files are edited; the patch lives in the fork branch.
 
## Existing tooling to reuse — do not reinvent
 
| tool | use |
|---|---|
| `/opt/ai/club-3090/scripts/p2p-validate.sh` | real NCCL all-reduce, checks **values**; exit 0/2/3/4/5 |
| `/opt/ai/club-3090/scripts/report.sh` | "Interconnect verdict" + "Transfer check" for the running container |
| `/opt/ai/club-3090/scripts/verify-full.sh` | ~2 min functional; step 8 degeneracy/cascade, step 9 MTP acceptance |
| `/opt/ai/club-3090/scripts/verify-stress.sh` | 10–20 min long-ctx needle ladder |
| `/opt/ai/club-3090/scripts/bench.sh` | the A/B harness behind the existing bench files |
| `/opt/ai/club-3090/scripts/launch.sh --variant vllm/qwen38-27b-dual-superfast` | boots the running slug (`dflash2-fp8.yml`) |
| `/opt/ai/club-3090/scripts/detect_nvlink.sh` | already handles patched-driver PCIe P2P in `auto`; `pcie_p2p` forces it |
| `~/p2p-check.sh` | byte-verified peer copy + GB/s, via the vLLM image's torch |
 
---
 
## Phase 1 — IOMMU passthrough (reboot #1)
 
Done **first**, on the current known-good 610.43.02 stack, so a passthrough problem is found
against a system whose behaviour you already know. Passthrough removes IOMMU DMA isolation for
all devices — fine on this trusted single-user box, not if VFIO guests ever get passthrough.
 
```bash
sudo sed -i 's/amd_iommu=on/iommu=pt/' /etc/default/grub
grep GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub    # eyeball before committing
sudo update-grub && sudo reboot
```
 
**GATE 1** — `/proc/cmdline` has `iommu=pt` · kernel logs `Default domain type: Passthrough` ·
no `Unknown option` line · both GPUs, NVMe and both NICs present · `topo -p2p` **still `GNS`**
(expected) · vLLM serves.
**Fail →** revert the sed, `update-grub`, reboot, and stop: without passthrough the patched
module reproduces August.
 
## Phase 2 — Deliberate kernel + driver upgrade (reboot #2)
 
Takes the upgrade that is coming anyway, at a moment you choose. 16 packages move together
(the whole userspace: `libnvidia-compute-610`, `-gl`, `-encode`, `-decode`, `nvidia-utils-610`,
firmware, …) plus the kernel to 7.0.0-31.
 
```bash
sudo apt update
apt list --upgradable | grep -E "nvidia|linux-image|linux-headers"   # read before committing
sudo apt install --only-upgrade $(dpkg -l | awk '/610\.43\.02/ {print $2}' | tr '\n' ' ')
sudo apt install linux-generic                                       # pulls -31 image + headers
sudo reboot
```
 
**GATE 2** — `nvidia-smi` reports **610.57.04** · `uname -r` is `7.0.0-31-generic` ·
`dkms status` shows stock `nvidia/610.57.04` built for the new kernel · `iommu=pt` survived ·
vLLM serves and `verify-full.sh` passes on the stock stack.
 
Then freeze it:
 
```bash
sudo apt-mark hold nvidia-dkms-610-open nvidia-driver-610-open \
                   nvidia-kernel-source-610-open nvidia-kernel-common-610
```
 
**Fail →** the previous kernel is still in the GRUB menu; boot it. The 610.43.02 rollback
tarballs in `~/p2p-run/rollback-cache/` are the only copy of that driver left.
 
## Phase 3 — Re-prime rollback, then baseline on the final stock stack
 
Rollback assets must describe the stack you are actually leaving. Unlike 610.43.02, this
version's .deb **is** downloadable — so cache the real packages, not just tarballs.
 
```bash
cd ~/p2p-run/rollback-cache
apt-get download nvidia-dkms-610-open nvidia-kernel-source-610-open nvidia-kernel-common-610
tar -czf stock-modules-$(uname -r).tar.gz \
    -C /var/lib/dkms/nvidia/610.57.04/$(uname -r)/x86_64 module
tar -czf stock-nvidia-610.57.04-src.tar.gz -C /usr/src nvidia-610.57.04
```
 
Then the seatbelt and the baseline. `detect_nvlink.sh` `auto` flips on one signal —
`topo -p2p r` reporting `OK` — and inspects neither IOMMU, ACS, BAR1, nor whether the grant
carries bytes. Pinning `force_off` decouples "the driver changed" from "serving changed",
which is exactly what went wrong in August.
 
```bash
echo "NVLINK_MODE=force_off" >> /opt/ai/club-3090/.env
[ -z "${NVLINK_MODE+x}" ] || echo "WARNING: NVLINK_MODE exported in shell — it WINS over .env (#425)"
cd ~/p2p-run
bash ~/p2p-run/rollback.sh status | tee state.before.txt
nvidia-smi topo -p2p rwnap > topo-p2p.before.txt
nvidia-smi --query-gpu=index,name,power.limit --format=csv > gpu-tune.before.txt
docker stop vllm-qwen38-27b-dual-superfast
bash ~/p2p-check.sh 2>&1 | tee bw.before.txt
bash /opt/ai/club-3090/scripts/p2p-validate.sh 2>&1 | tee validate.before.txt; echo "exit=$?"
docker start vllm-qwen38-27b-dual-superfast      # wait for ready
bash /opt/ai/club-3090/scripts/verify-full.sh 2>&1 | tee verify.before.txt
bash /opt/ai/club-3090/scripts/bench.sh 2>&1 | tee ~/bench-nop2p-610.57.04-$(date +%F).txt
```
 
**GATE 3** — debs downloaded and new tarballs written · `.env` has `force_off` and
`NVLINK_MODE` is not exported in the shell · `GNS` confirmed · bandwidth recorded (expect
~4–6 GB/s) · `p2p-validate.sh` exit **0** on stock · `verify-full` passes **including step 9
MTP ≥ 2.0** (the pre-patch reading that makes the post-patch one meaningful) · fresh bench
captured on 610.57.04.
 
## Phase 4 — Build the patched modules (nothing installed)
 
The clone has ~1498 root-owned objects from August and is on the old branch; clean before
switching or the checkout fights you.
 
```bash
cd ~/open-gpu-kernel-modules
sudo make clean && sudo chown -R leo3090:leo3090 .
git fetch origin && git checkout 610.57.04-p2p-v3 && git status -sb
grep -n "ENABLE_RESIZABLE_BAR, 1" kernel-open/nvidia/nv-reg.h    # confirm the patch is present
make modules -j16 2>&1 | tee ~/p2p-run/build.log
```
 
**GATE 4** — all five modules present with `version: 610.57.04`, `vermagic: 7.0.0-31-generic`,
`license: Dual MIT/GPL`, no modpost errors. A version mismatch here is the §8 "match the patch
branch to your exact driver version" row — the single most common cause of a hollow grant.
 
A dry run of what DKMS does in Phase 5; failure here is free.
 
## Phase 5 — Install via DKMS (reboot #3)
 
```bash
docker stop vllm-qwen38-27b-dual-superfast
sudo systemctl stop nvidia-persistenced        # it holds the driver open
sudo bash ~/p2p-run/install-dkms.sh            # after the version bump noted in Prep
```
 
The script stages a clean copy to `/usr/src/nvidia-610.57.04-p2p`, runs `dkms add` and
`dkms build`, **verifies the built artifacts, and only then** removes stock and installs — so a
build failure leaves the running system fully intact.
 
**GATE 5 (pre-reboot, the script prints it)** — every `modinfo -n` under `updates/dkms/` ·
`version 610.57.04`, `vermagic 7.0.0-31-generic` · `dkms status` shows `nvidia/610.57.04-p2p`
installed and **no** stock `nvidia/610.57.04`.
 
**Fail →** Route B in the same sitting:
`sudo dkms remove -m nvidia -v 610.57.04 --all && cd ~/open-gpu-kernel-modules && sudo make modules_install && sudo depmod -a`,
accepting that kernel upgrades then need Phases 4–5 rerun by hand.
 
**PASS →** `sudo reboot`
 
## Phase 6 — Prove the grant carries bytes (serving still pinned off)
 
Walk the §7a correctness ladder before anything serving depends on it.
 
```bash
nvidia-smi topo -p2p rwnap | tee ~/p2p-run/topo-p2p.after.txt
journalctl -k -b | grep -iE "NVRM|Xid"
bash ~/p2p-check.sh 2>&1 | tee ~/p2p-run/bw.after.txt
bash /opt/ai/club-3090/scripts/p2p-validate.sh 2>&1 | tee ~/p2p-run/validate.after.txt; echo "exit=$?"
```
 
**GATE 6** — `OK` not `GNS` · no Xid · BAR1 still 32768 MiB · peer copy **correctness PASS**
with bandwidth clearly above Phase 3 (expect ~8–13 GB/s) · **`p2p-validate.sh` exit 0**.
 
| exit | action |
|---|---|
| 2 hang | hollow grant. Confirm `iommu=pt` took; check ACS on `00:01.1`/`00:01.3`; only then consider §5's `NVreg_RegistryDwords` — **Blackwell evidence, 3090 is confirmed working without it, and it costs ~3% when P2P is off** |
| 3 wrong data | hard stop, `rollback.sh l1`, do not proceed |
| 4/5 | harness problem — compare against `validate.before.txt` |
 
## Phase 7 — Serving on P2P, gated on output
 
```bash
docker rm -f vllm-qwen38-27b-dual-superfast        # drops the layer holding the stale cache
find /opt/ai/club-3090 -name "gpu_p2p_access_cache_for_*.json" -delete
sed -i 's/^NVLINK_MODE=.*/NVLINK_MODE=pcie_p2p/' /opt/ai/club-3090/.env
cd /opt/ai/club-3090
VLLM_SKIP_P2P_CHECK=0 bash scripts/launch.sh --variant vllm/qwen38-27b-dual-superfast
bash scripts/report.sh        2>&1 | tee ~/p2p-run/report.after.txt
bash scripts/verify-full.sh   2>&1 | tee ~/p2p-run/verify.after.txt
bash scripts/verify-stress.sh 2>&1 | tee ~/p2p-run/stress.after.txt      # 10-20 min
```
 
The vLLM P2P cache is keyed **only** on the device-id list — nothing that moves when the driver
changes — so a `true` written before the swap would outlive it.
 
**GATE 7** — weights actually load (a hang at `pynccl` with `vLLM is using nccl==…` as the last
line is the §8 signature → `rollback.sh l0`, return to Gate 6) · `report.sh` verdict
`✓ engaged` · Transfer check `✓ N/N directed pairs OK` · `verify-full` passes **every** step,
especially step 8 degeneracy and **step 9 MTP ≥ 2.0** · `verify-stress` passes.
 
**If step 8 or 9 fails you are in #922** — transport fine, custom kernel not. Do not roll back
the driver. Force `AR="--disable-custom-all-reduce"` at
`/opt/ai/club-3090/models/qwen3.8-27b/vllm/compose/dual/autoround-int4/dflash2-fp8.yml:176`,
relaunch, re-verify. That keeps the prefill win, which §6 calls the reliable half.
 
## Phase 8 — Measure and decide
 
```bash
bash /opt/ai/club-3090/scripts/bench.sh 2>&1 | tee ~/bench-p2p-610.57.04-$(date +%F).txt
```
 
Compare against the **Phase 3** file — same driver, same kernel, same sitting. Do not compare
against `bench-baseline-nop2p.txt`; §6 records a cross-session baseline drifting 17.7%.
 
**GATE 8** — no metric regressed beyond noise · prefill @10K and @90K improved (if not, P2P is
not carrying the all-reduce — re-read the verdict before believing the bench) · power caps
identical across arms (`nvidia-pl.service` re-applied the 250 W / 220 W limits recorded in
`gpu-tune.before.txt`; a service that failed to re-apply invalidates the comparison).
 
Read prefill first. Decode gains come from the custom kernel, not the link.
 
---
 
## Rollback
 
```bash
bash ~/p2p-run/rollback.sh status      # no sudo; prints state and recommends a tier
```
 
| tier | when | cost |
|---|---|---|
| **L0** | hang at `pynccl`, or #922 garbage — driver is fine | ~30 s, **no reboot** |
| **L1** | stock driver back, kernel unchanged | ~2 min + reboot, no compile, no network |
| **L2** | stock driver back, kernel has moved | ~15 min + reboot |
| **L3** | trouble that began with `iommu=pt` | reboot |
| **L4** | will not boot — GRUB `e`, drop `iommu=pt`, Ctrl-X | manual |
 
L0 covers both likely failures. L1's guard compares snapshot vermagic against the running
kernel and refuses, pointing at L2, if they differ.
 
Going back further than 610.57.04 is possible but one-way-ish: its .deb is fetchable, 610.43.02's
is **not**. The only remaining copy of 610.43.02 is the pair of tarballs already in
`~/p2p-run/rollback-cache/` — do not delete them.
 
## Verification (end to end)
 
1. `bash ~/p2p-run/rollback.sh status` → module under `updates/dkms`, `dkms status` shows only
   `nvidia/610.57.04-p2p`, `iommu: pt`, `NVLINK_MODE=pcie_p2p`
2. `nvidia-smi topo -p2p rwnap` → `OK` both directions
3. `bash ~/p2p-check.sh` → correctness PASS, bandwidth ≫ `bw.before.txt`
4. `bash /opt/ai/club-3090/scripts/p2p-validate.sh` → exit 0
5. `bash /opt/ai/club-3090/scripts/report.sh` → `✓ engaged` + `✓ N/N directed pairs OK`
6. `bash /opt/ai/club-3090/scripts/verify-full.sh` → all steps, step 9 MTP ≥ 2.0
7. `bash /opt/ai/club-3090/scripts/verify-stress.sh` → needle ladder passes
8. `bash /opt/ai/club-3090/scripts/bench.sh` → prefill up vs the Phase 3 file, decode not down
9. **The DKMS payoff:** at the next kernel bump, confirm `dkms status` rebuilt
   `nvidia/610.57.04-p2p` automatically and `topo -p2p` still reports `OK` after reboot
 
## Known risks
 
- **`install-dkms.sh` is unrun**, and now also needs its version constants bumped. The `MAKE[0]`
  line is reasoned from the fork's two Makefiles plus Ubuntu's known-good conf, not observed —
  no passwordless sudo here. It fails at build time with stock intact if wrong.
- **610.57.04 is untested on this box**, as is the `-v3` branch. Its README is the most developed
  of the three variants (RTX 30/40/50, mixed-generation, shared console mapping) and the base
  `610.57.04-p2p` branch dates from 2026-08-13, so the line has had a month of iteration — but
  nobody here has run it.
- **#922 on AMD is unknown** — reproduced on two Intel rigs, not on Threadripper. Gate 7 is the
  mitigation; `verify-full` step 9 is a direct detector on this slug.
- **`verify-full` cannot catch fluent-but-wrong output** — hence `verify-stress` in Gate 7.
- **ACS is unread** (needs root). If bandwidth disappoints with everything else green, that is
  the first suspect; Ubuntu's generic kernel has no `pcie_acs_override`, so it is a BIOS setting.
- **Holds are a commitment.** Unholding and upgrading past 610.57.04 requires a matching fork
  branch to exist first, or you get the hollow-grant hang.

La violence de la longueur du poste au dessus. Tu as gagné en perf sinon ?

Ça me saoule, je voudrais bien tout poster. Je vais pétre faire un GitHub pour y mettre les scripts.

J'attends la, ça bench...

Mais sinon le test de bw pcie inter carte a fait X2.. 6.5GBPS -> 13

Bon, pour l'instant, c'est toujours aussi "stable" depuis 3j, pas de crash du driver et analyse en cours :
 
5675.01.062.602 I slot print_timing: id  0 | task 233273 | prompt eval time =   27587.41 ms / 10826 tokens (    2.55 ms per token,   392.43 tokens per second)
5675.01.062.607 I slot print_timing: id  0 | task 233273 |        eval time = 1135546.02 ms / 32768 tokens (   34.66 ms per token,    28.86 tokens per second)
5675.01.062.608 I slot print_timing: id  0 | task 233273 |       total time = 1163133.43 ms / 43594 tokens
5675.01.062.609 I slot print_timing: id  0 | task 233273 |    graphs reused =     240502
5675.01.062.613 I slot print_timing: id  0 | task 233273 | draft acceptance = 0.69298 (22124 accepted / 31926 generated), mean len =  3.0
 
qwen3.8-27b-gsq-rco kv q8/q8 max context

ce post pour tester mon script markdown pour hfr

 
Ha si t'en a un je prends, j'ai essaye vite fait sans success.

 
Hm malheureusement la box redemarre des que l'inference commence quand le p2p est active... J'enquete...

J'en ai un oui mais il marche pas encore super bien sur ton post, il le fait déjà à moitié c'est un début

 
Il dit quoi sudo journalctl -k -b -1 -e ?

• Ryzen 7 7700 (AM5), 30 GB RAM
• RTX 3090 Ti + RTX 3090 — both Gen4 x8, separate CPU root ports (topo = PHB), no NVLink possible (Ti has no fingers)
• Ubuntu 26.04, kernel 7.0.0-31-generic
• Driver 610.57.04 + aikitoria/open-gpu-kernel-modules branch 610.57.04-p2p-v3, installed via DKMS
• iommu=pt, Resizable BAR on (BAR1 = 32768 MiB/GPU), Secure Boot off
• Power limits 250 W / 220 W, core +100, mem +300/+400 (long-standing stable OC)
• vLLM 0.29.0, Qwen3.8-27B int4, TP=2, fp8 KV, DFlash2 drafter n=7, 262K ctx, gpu-memory-utilization 0.93

P2P came up clean and verified

Conclusion

On this rig the peer transport is stable and worth having; vLLM's custom all-reduce kernel over BAR1 P2P is not. Running P2P with --disable-custom-all-reduce keeps the prefill/TTFT win and the machine stays up.

This looks like the same component implicated in the known "custom AR returns garbage over patched P2P" reports (two Intel rigs, wrong output + 0% MTP accept), but with a different failure mode: on AM5 with iommu=pt it takes the whole machine down instead. Worth noting that iommu=pt removes DMA isolation, so a malformed peer write has nothing to catch it — consistent with an instant reset and zero logs.

 
Les 3090 c'est les fameuses cartes qui supportent le NVLINK ? Dans ce cas, pourquoi tu t'emmerdes avec le P2P PCI-E, ca sera encore plus performant que tu utilises NVLINK

 
Ben c'est a dire que j'ai une 3090 Ti, qui n'a pas de NVLINK...  
 
Et...
 
Un pont NVLINK pour generation Ada / 3090 coute 900-1200 balles  

 
Aliexpress  
 
Bon après si t'as pas le port ca sert à rien

 
Il te reste pas un peu de scotch?
 

Ça c'est une bien belle installation comme on aimerait en voir plus souvent  

 
C'est quoi ce truc ?  

 
C'etait les prix aliexpress que j'ai vu lol C'est devenu completement wtf.

 
Ben un tuto pour toi
 

J'ai retrouvé un ASUS GX10 1To à 4600 euros ! Et 2to à 4800 euros
vs Mac Studio M5 Max 128Go / 1To à 5600 (edu).
 
   
 
 
 
Le GX10 devance largement le Mac sur l'usage agentique / multi-users / MoE.
Kiffe-kiffe sur un modèle dense, flux unique.
 
Le mac reste un vrai desktop, avec capacité de montage vidéo & apps diverses.
 
 

FOMO  

A ce rythme là on aura des annonces LBC a base de "échange 208 contre mac studio"

Un challenge : combien de voiture je peux m'achater avec un DGX Spark

 
j'échange trois serveurs IA contre ma maison  

 
3 BC250 ça comptes?

Question de merde.
On peut installer un LLM sur windows sans aucune connaissance ?
Genre comme un logiciel de chat, où l'on aurait juste à charger des modèles pour voir ce que ca donne  

Question bis : Comment on dimensionne un modèle pour un usage autrement que de façon empirique?
Et comment on le spécialise ?

Oui, par exemple LM Studio, ça se fait en trois clics.

LMstudio (avec ses drivers pré intégrés) ou llama une fois les drivers installés (ROCm / Cuda / Vulkan ...). Moi j'aime bien l'interface et la facilité de LMStudio mais attention c'est pas complètement de l'Open Source Software
tu dimensionnes un modèle selon ton hardware. Les 27b / 35b (qui fitent bien dans 32GB) sont quand même devenus très performants. Tu peux gratter un peu en allant sur un DGX / Halo Strix 128GB mais le gain est pas franchement énorme. Ca redevient intéressant de grater avec 396 / 512GB de VRAM mais les tarifs   font qu'au final y'a aucun monde où tu rentabilises le bordel autrement qu'en entreprise (il me faut 27 mois pour produire autant de tokens à la maison que ce que j'aurai pu obtenir pour le même prix chez deepseek en flash 4.1 qui est bien meilleur que ce que je peux auto heberger en prime ...
 
Par contre on me modifie pas mes prix sans prévenir, j'ai pas de problème d'indispo si j'ai pas de panne et mes données ne sont pas analysées / ou requérables par un état étranger sans que j'en sois informé : elles restent chez moi
 
Pour le spécialiser tu as la methode du puriste : créer un RAG https://superml.org/tutorials/rag-beginner ou simplement sélectionner quelques documents "représentatifs" que tu ajoutes en iput à chaque requète avec l'instruction en te bansant le document A et B corrige/modifie/créé le document C

 
dans quelques mois la valeur va être multipliée par 10, ça se pourrait bien    

 
This    

 
étape 1: est-ce que tu as un GPU?

Merci pour les réponses    
 
J’ai une 5060 8Go  

combien de RAM ? Tu peux jouer avec de l'offloading CPU sur un modèle MoE (Qwen 3.6 35B a3b par exemple). Ce sera pas super rapide par contre  

Je voulais me payer une deuxieme 3090.
 
Et j'ai vu le prix sur leboncoin qui est dingue

 
Ouais, quand j'ai raqué pour une 3090 FE à 1500 balles, je m'étais dit qu'à l'époque c'était un caprice, mais six ans plus tard, elle tape encore huit cent balles d'occaz, elle reste un incontournable des cartes grand public pour s'initier à l'IA locale et avec le mod MFG il y a de quoi lui donner une seconde vie en jeu.

 
 
 
Ça devrait aller, quand ça fonctionne bien j'ai environ 16 à 22 tokens par seconde que ce soit sur LM STUDIO ou en mode terminal via Llama server.
 
Un point cependant pour LM STUDIO le moteur de base llama.cpp V2.39.0 utilisé par LM STUDIO la version du moteur llama.cpp c'est en fait b10987 (commit d1d3c3396)
hors elle date un peu un on est actuellement en B10997 sur Github: https://github.com/ggml-org/llama.c [...] ase-b10997
Autre point : une version finale Llama.cpp est sortie c'est la V0.4.1 https://github.com/ggml-org/llama.cpp/tree/v0.4.1
 
Version Terminal Adminstateur qui s'installe en mode terminal via la commande suivante:
 
winget install llama.cpp
 
Via le Shell en mode Administrateur.
 
La version Terminal est indépendante de LM STUDIO c'est carrément un autre logiciel et très relou à configurer, et pour s'en servir ça démarre une page via le hoste en 127.  avec un port en 8080 comme ceci: http://127.0.0.1:8080/ et là tu causes à l'IA en écrivant en langage naturel dans la langue de ton choix, tu peux même écrire en plusieurs langues dans le même prompt.
 
 
Au fait c'est très béta comme machin, ça peut crashé ta carte graphique auquel cas après deux ou trois écran noir je te recommande vivement de réinstaller les drivers de ta carte graphique car ça les abimes ce qui donne lieu à encore plus de crash et d'écran noir.

Je peux arriver à faire travailler différent GPU en même temps? genre une 4060 TI et une 2080 ?  

vi faut soigner la répartition des layers mais t'as plusieurs spécialistes des architectures cheloues hybrides sur le topic

Bien sûr, de mon côté je tourne avec 3 GPU : 2 3060 12G + 1 2070 8Go
J'utilise llama.cpp pour faire tourner Qwen 3.8 27b UD_Q4_XL avec 192k de contexte

J'ai simplement filé ma config de llama-server au modèle, je lui ai décrit mon hardware, mon besoin et on a trouvé le meilleur compromis vitesse/contexte pour moi
après quelques itérations et tests j'ai gagné 25% de tg et 65k de contexte par rapport à ce que j'avais paramétré seul de mon côté notamment en jouant sur le tensor-split pour utiliser le max de VRAM de chaque GPU (et en virant quelques arguments bidons)

J'arrive à 12-13 tk/s tg et environ 500 en pp (avec contexte vide, ça diminue progressivement quand le contexte se rempli mais largement utilisable)

La conf pour  info :

 
Perso j'utilise Unsloth : https://github.com/unslothai/unsloth
Il est connu pour son fine tuning de modèle mais il fait très bien ce que tu demandes :
- tu l'installes
- tu choisis un modèle (il y a moyen de lui demander les modèles qui tiennent sur ta config et tu auras un boule verte, un "tight" ou un "OOM" (out of memory))
- tu attends qu'il d/l le modèle
- tu peux jouer avec
 
Je n'utilise pas LM, donc je ne peux pas comparer, mais Unsloth est à ma connaissance open-source si c'est un critère.
 
Il est encore en beta et les mises à jour sont fréquentes. Mais il tourne bien.