Self-referenced type restriction on inferred type argument in Typescript

Question

I want to have an object builder that should infer types similar to how zod.js does.

It should receive fallbacks as an argument and return an object to work with. Fallback could be a value or a function that returns a value. That function should also be able to use other fields as an argument.

In a simple case it should be something like this:

type TypeOf<T> =
    T extends (...args: any) => any ? TypeOf<ReturnType<T>> | null : 
    T extends object ? { [K in keyof T]: TypeOf<T[K]> } : T;

type Without<T, K> = Pick<T, Exclude<keyof T, K>>;

type Fields<T> = {
    [K in keyof T]: string | (<A extends Without<T, K>>(arg: A) => string);
};

declare function build<T>(fallbacks: Fields<T>) : TypeOf<T>;

// `obj` should have inferred type `{ simple: string, fallback: string }`
const obj = build({
    simple: "test",
    fallback: (obj) => `hello, ${obj.simple}`, //  <--  obj has unknown type
});

console.log(obj); // { simple: "test", fallback: "hello, test" }

obj.simple = "world";
console.log(obj); // { simple: "world", fallback: "hello, world" }

obj.fallback = "custom"; 
console.log(obj); // { simple: "world", fallback: "custom" }

obj.fallback = null; 
console.log(obj); // { simple: "world", fallback: "hello, string" }

It is working if I strictly define my type:

type MyType = {
    simple: string
    fallback: string | null
}

const obj = build<MyType>({ ... });

Or if I remove constrictions on the fallback function argument:

type Fields<T> = {
    [K in keyof T]: string | ((arg: any) => string);
};

So, either type inference not working or the fallback function argument isn't well typed.

As I understand, the main issue is the type argument could be in "inference" mode that resolves types from what it has or in "restriction" mode that checks values by passed types.

Is there a way to have a partial self-referenced type restriction on inferred type argument?