feat: compliant cli checker for correct variants and dependencies #121

drbh · 2025-04-14T15:23:41Z

This PR adds a new cli tool compliant that checks kernels for compliance.

$ compliant
Hugging Face kernel compliance checker

Usage: compliant <COMMAND>

Commands:
  list   List fetched repositories with build variants
  check  Check repository compliance and ABI compatibility
  help   Print this message or the help of the given subcommand(s)

Options:
  -h, --help     Print help
  -V, --version  Print version

check args

$ compliant check --help
Check repository compliance and ABI compatibility

Usage: compliant check [OPTIONS] --repos <REPOS>

Options:
  -r, --repos <REPOS>            Repository IDs or names (comma-separated)
  -m, --manylinux <MANYLINUX>    Manylinux version to check against [default: manylinux_2_28]
  -p, --python-abi <PYTHON_ABI>  Python ABI version to check against [default: 3.9]
  -a, --auto-fetch               Automatically fetch repositories if not found locally
  -r, --revision <REVISION>      Revision (branch, tag, or commit hash) to use when fetching [default: main]
      --long                     Show all variants in a long format. Default is compact output
      --show-violations          Show ABI violations in the output. Default is to only show compatibility status
      --format <FORMAT>          Format of the output. Default is console [default: console] [possible values: console, json]
  -h, --help                     Print help

Usage

list all kernels (in cache and have a build variant)

$ compliant list
.
├── kernels-community/activation
├── kernels-community/deformable-detr
├── kernels-community/flash-mla
├── kernels-community/quantization
╰── 4 kernel repositories found

checking a repo

 $ kernels-community/activation
├── build: Total: 13 (CUDA: 12, ROCM: 1)
│   ├── ✓ CUDA
│   ╰── ✗ ROCM
╰── abi: compatible
    ├── ✓ manylinux_2_28
    ╰── ✓ python 3.9

with --long output for variant specific abi compatibility

$ compliant check --repos kernels-community/activation --long
├── build: Total: 13 (CUDA: 12, ROCM: 1)
│  ✓ CUDA
│    ├── torch25-cxx11-cu118-x86_64-linux
│    ├── torch25-cxx11-cu121-x86_64-linux
│    ├── torch25-cxx11-cu124-x86_64-linux
│    ├── torch25-cxx98-cu118-x86_64-linux
│    ├── torch25-cxx98-cu121-x86_64-linux
│    ├── torch25-cxx98-cu124-x86_64-linux
│    ├── torch26-cxx11-cu118-x86_64-linux
│    ├── torch26-cxx11-cu124-x86_64-linux
│    ├── torch26-cxx11-cu126-x86_64-linux
│    ├── torch26-cxx98-cu118-x86_64-linux
│    ├── torch26-cxx98-cu124-x86_64-linux
│    ╰── torch26-cxx98-cu126-x86_64-linux
│  ✗ ROCM
│    ├── torch25-cxx11-rocm5.4-x86_64-linux
│    ├── torch25-cxx11-rocm5.6-x86_64-linux
│    ├── torch25-cxx98-rocm5.4-x86_64-linux
│    ├── torch25-cxx98-rocm5.6-x86_64-linux
│    ├── torch26-cxx11-rocm5.4-x86_64-linux
│    ├── torch26-cxx11-rocm5.6-x86_64-linux
│    ╰── torch26-cxx11-rocm62-x86_64-linux
╰── abi: compatible
    ├── ✓ manylinux_2_28
    ╰── ✓ python 3.9

screenshot to show coloring

Checks

must contain all builds for a given arch (cuda/rocm)
must pass abi-check
conforms to having a build dir
add json output format

…ole formatting

danieldk

Really nice to have this! Added a bunch of comments.

danieldk · 2025-04-24T10:47:04Z

compliant/Cargo.toml

@@ -0,0 +1,29 @@
+[package]
+name = "compliant"


Maybe we should name it kernel-compliance-check? A bit more descriptive.

agreed, updated dir and tool name in latest changes. Thanks!

danieldk · 2025-04-24T10:50:34Z

compliant/src/lib.rs

+use anyhow::{Context, Result};
+use clap::{Parser, Subcommand, ValueEnum};
+use colored::Colorize;
+use hf_hub::api::tokio::{ApiBuilder, ApiError};
+use hf_hub::{Repo, RepoType};
+use kernel_abi_check::{check_manylinux, check_python_abi, Version};
+use object::Object;
+use once_cell::sync::Lazy;
+use serde::{Deserialize, Serialize};
+use std::fmt;
+use std::fs;
+use std::path::{Path, PathBuf};
+use thiserror::Error;


Suggested change

use anyhow::{Context, Result};

use clap::{Parser, Subcommand, ValueEnum};

use colored::Colorize;

use hf_hub::api::tokio::{ApiBuilder, ApiError};

use hf_hub::{Repo, RepoType};

use kernel_abi_check::{check_manylinux, check_python_abi, Version};

use object::Object;

use once_cell::sync::Lazy;

use serde::{Deserialize, Serialize};

use std::fmt;

use std::fs;

use std::path::{Path, PathBuf};

use thiserror::Error;

use std::fmt;

use std::fs;

use std::path::{Path, PathBuf};

use anyhow::{Context, Result};

use clap::{Parser, Subcommand, ValueEnum};

use colored::Colorize;

use hf_hub::api::tokio::{ApiBuilder, ApiError};

use hf_hub::{Repo, RepoType};

use kernel_abi_check::{check_manylinux, check_python_abi, Version};

use object::Object;

use once_cell::sync::Lazy;

use serde::{Deserialize, Serialize};

use thiserror::Error;

I like this separation of imports between stdlib <-> external crates <-> local.

oh yea thats better, updated in latest changes. Thanks

danieldk · 2025-04-24T10:52:43Z

compliant/src/lib.rs

+        /// Automatically fetch repositories if not found locally
+        #[arg(short, long, default_value = "true")]
+        auto_fetch: bool,


I think it'd be nice if auto-fetching is the default:

If cached and up-to-date, immediately use local.

Otherwise download and use local.

removed and improved to always check if the local source is the latest hash of that revision and added and sync if needed. Also added --force to enable fetching even if you have the copy locally

danieldk · 2025-04-24T10:55:03Z

compliant/src/lib.rs

+#[derive(Subcommand)]
+pub enum Commands {
+    /// List fetched repositories with build variants
+    List {


Do we need this? I like that with normal Hub/kernels usage there is not a real distinction between local/remote. You give a repo name and the library takes care of everything. With list and non-auto download (see below), you have to think about where things live, etc.

yea that makes sense, removed in latest commits

danieldk · 2025-04-24T10:57:35Z

compliant/src/lib.rs

+}
+
+async fn fetch_compliant_variants() -> Result<(Vec<String>, Vec<String>)> {
+    let url = "https://raw.githubusercontent.com/huggingface/kernel-builder/refs/heads/main/build-variants.json";


Maybe should move this build-variants.json into the src directory of the checker and bake it into the binary? I think this allows more control over future changes of the JSON file and avoids breaking all past versions if the format changes.

I think this makes alot of sense. For now i've added a new build.rs script that vendors the json in a rs file at build time so its baked into the binary. I think this should add alot more stability and avoids fetching the variants during each run

danieldk · 2025-04-24T11:14:33Z

compliant/src/lib.rs

+    let hash = content.trim();
+    let snapshot_dir = repo_path.join(format!("snapshots/{}", hash));


I think we can directly get the correct repo directory from hf-hub, so that we don't have to care about the on-disk representation?

agreed, this has been simplified to prefer the hf-hub in latest commits

danieldk · 2025-04-24T11:17:10Z

compliant/src/lib.rs

+    let repo_path = get_repo_path(repo_id, cache_dir);
+
+    // Check if repository exists locally
+    if !repo_path.exists() || !repo_path.join("refs/main").exists() {


Fails on other branches.

now relying on hf-hub

danieldk · 2025-04-24T11:17:25Z

compliant/src/lib.rs

+    }
+
+    // Re-check after potential fetch
+    let ref_file = repo_path.join("refs/main");


Fails on other branches.

now relying on hf-hub

danieldk · 2025-04-24T11:17:58Z

compliant/src/lib.rs

+        .with_context(|| format!("Failed to read ref file: {:?}", ref_file))?;
+
+    let hash = content.trim();
+    let snapshot_dir = repo_path.join(format!("snapshots/{}", hash));


Get the directory from hf-hub.

now relying on hf-hub

danieldk · 2025-04-24T11:20:48Z

compliant/src/main.rs

@@ -0,0 +1,155 @@
+use anyhow::{Context, Result};


Small nit: we use eyre in build2cmake and kernel-abi-check. Maybe we should use the same here for consistency?

good catch, i've updated to prefer eyre over anyhow in the latest changes. Thanks!

feat: compliant cli checker for correct variants and dependencies

bd4f49f

drbh force-pushed the add-compliant-checker-tool branch from bcc946d to bd4f49f Compare April 14, 2025 22:52

drbh added 8 commits April 14, 2025 22:45

feat: improve output and add json format option

d39e10e

fix: explore various download methods

6425776

fix: simplify fetch logic to single path

9b61b02

fix: fetch variants from repo

0074903

feat: split files and simplify formatting

bfa9f58

feat: refactor logic into lib, improve error handling, types and cons…

beb75ec

…ole formatting

fix: prefer versioned kernel-abi-check dep

d2fec1c

fix: improve metadata to prep for cargo publish

6eaaf81

drbh marked this pull request as ready for review April 24, 2025 00:07

danieldk reviewed Apr 24, 2025

View reviewed changes

drbh added 11 commits April 28, 2025 12:48

feat: rename project and dir

adf5eb1

fix: split dep imports

6f34d1e

feat: improve core sync logic and file structure

6c1d3f4

feat: vendor variants at build time

513be21

feat: prefer eyre over anyhow

32d2f53

fix: remove unused struct

36c2905

fix: prefer peekable over manual enumeration

a1cec6a

fix: add missing eyre dep change

fdfefcc

feat: check hub for latest snapshot in all cases

4e8ec3f

fix: address all clippy lints

c15151b

fix: small typo cleanup

40d8da1

drbh requested a review from danieldk May 1, 2025 13:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: compliant cli checker for correct variants and dependencies #121

feat: compliant cli checker for correct variants and dependencies #121

drbh commented Apr 14, 2025 •

edited

Loading

danieldk left a comment

danieldk Apr 24, 2025

drbh Apr 29, 2025

danieldk Apr 24, 2025

drbh Apr 29, 2025

danieldk Apr 24, 2025

drbh Apr 29, 2025

danieldk Apr 24, 2025

drbh Apr 29, 2025

danieldk Apr 24, 2025

drbh Apr 29, 2025

danieldk Apr 24, 2025

drbh Apr 29, 2025

danieldk Apr 24, 2025

drbh Apr 29, 2025

danieldk Apr 24, 2025

drbh Apr 29, 2025

danieldk Apr 24, 2025

drbh Apr 29, 2025

danieldk Apr 24, 2025

drbh Apr 29, 2025

		let hash = content.trim();
		let snapshot_dir = repo_path.join(format!("snapshots/{}", hash));

feat: compliant cli checker for correct variants and dependencies #121

Are you sure you want to change the base?

feat: compliant cli checker for correct variants and dependencies #121

Conversation

drbh commented Apr 14, 2025 • edited Loading

Usage

Checks

danieldk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

drbh commented Apr 14, 2025 •

edited

Loading