feat(collections): create from and read raw schema JSON

[dudanogueira]

What

Adds a raw-JSON escape hatch to the collections API, in both directions, on the sync and async clients:

client.collections.createFromJson(json);      // POST /schema
client.collections.getConfigAsJson(name);     // GET  /schema/{name}  -> Optional<String>
client.collections.listAsJson();              // GET  /schema        -> String

Neither direction maps the document onto CollectionConfig. On write the body is forwarded byte-for-byte, so any option the server accepts works — including ones this client version does not model yet. On read the response body is returned untouched.

Why the read side is here too

This started as write-only, but the read path turned out to need the same escape hatch: CollectionConfig deserialization is lossy today, so there is currently no way to render or diff a real collection faithfully using the client alone.

The most visible case: VectorConfig.CustomTypeAdapterFactory.read looks for the bq/pq/sq/rq keys at the top of vectorIndexConfig. That is correct for hnsw and flat, but a dynamic index nests them one level deeper under hnsw/flat. Verified against a live 1.38.0 collection with a dynamic index and RQ enabled:

raw     hnsw.rq = {"bits":8,"enabled":true,"rescoreLimit":20}
typed   quantization = null

Same root cause loses Dynamic.flat's own quantizers.

Also dropped on read: property-level moduleConfig (the server's actual home for skip / vectorizePropertyName), moduleConfig entries that are neither reranker-* nor generative-*, the read-only shardingConfig fields (actualCount, function, key, strategy), and vectorIndexConfig.distance at the dynamic level.

getConfigAsJson unblocks callers who need the complete picture without waiting for the mapping to be fixed.

Design notes

• createFromJson reads exactly one key, "class", which it needs in order to return a collection handle. A document that is not a JSON object, or that has no non-empty string "class", raises IllegalArgumentException before the request is sent rather than after a round-trip.
• getConfigAsJson validates nothing — there is no name to extract — and maps 404 to Optional.empty(), matching the typed getConfig.
• create(String) already means "create by collection name", so the new method could not be an overload; hence the distinct names.

Not addressed here

Two parsing bugs found while investigating, left for follow-ups so this PR stays additive:

1. Dynamic-index quantization (above). Needs a design call before fixing: under dynamic, hnsw and flat can carry different quantizers, but VectorConfig has a single quantization() slot.
2. baseURL vs baseUrl. 28 classes use @SerializedName("baseURL"). Probing the server directly, sending baseURL gets echoed back as an unrecognized passthrough key alongside the module's real baseUrl default, while sending baseUrl actually takes effect — so the base URL is neither written effectively nor read back. rerankers/NvidiaReranker.java already uses baseUrl. Only text2vec-cohere was verified empirically; the canonical key should be confirmed per module before any blanket rename.

Testing

• 13 new unit tests covering endpoint shape, verbatim body passthrough in both directions, 404 → empty, and every createFromJson validation path.
• Two new cases in CollectionsITest exercising create-from-JSON → getConfigAsJson → listAsJson → delete, using a dynamic-index-with-RQ payload.

Full unit suite passes. The integration tests could not run locally (testcontainers fails to initialize in my environment — all pre-existing ITs error the same way), so the new integration test logic was additionally verified by running an identical copy against a live Weaviate 1.38.0: the collection was created from raw JSON, getConfigAsJson returned hnsw.rq.enabled = true intact, listAsJson saw it, and the post-delete lookup returned empty. CI will be the real check on the ITs.

[orca-security-eu]

Orca Security Scan Summary

Status	Check	Issues by priority
Passed	Infrastructure as Code	0   0   0   0	View in Orca
Passed	SAST	0   0   0   0	View in Orca
Passed	Secrets	0   0   0   0	View in Orca
Passed	Vulnerabilities	0   0   0   0	View in Orca

[bevzzz]

I suggest a different approach, probably one that will result in better code-reuse / less duplication.

The only difference between create(CollectionConfig) and createFromJson(String) is the payload. To add "from/to JSON" to the current client it should be enough to allow GetConfigRequest, CreateCollectionRequest, and ListCollectionsRequest to accept a generic parameter saying what the request/response payload should be:

public record GetConfigRequest(String collectionName) {
  public static final <T> Endpoint<GetConfigRequest, Optional<T>> endpoint(Class<T> cls) {
    return OptionalEndpoint
        .<GetConfigRequest, T>noBodyOptional(
            request -> "GET",
            request -> "/schema/" + request.collectionName,
            request -> Collections.emptyMap(),
            (statusCode, response) -> JSON.deserialize(response, cls));
  }
}

This way the same GetConfigRequest can be used to return:

• CollectionConfig, if used like so GetConfigRequest.endpoint(CollectionConfig.class)
• Raw JSON string, if used like so GetConfigRequest.endoint(String.class)

Same for list and create endpoints.

Would you like to take a stab at that? I'm happy to take this over, this seems like a small enough change.

---

Wrt to the "Not addressed here" section: if you find any bugs while working on a PR, please open a ticket for those. A PR description is not the right place to document bugs.