![]() Var imageResultTensor = VaeDecoder.Decoder(decoderInput) Var unetSession = new InferenceSession(modelPath, options) įor (int t = 0 t Var latents = GenerateLatentSample(batchSize, height, width, seed, scheduler.InitNoiseSigma) Var timesteps = scheduler.SetTimesteps(numInferenceSteps) Var scheduler = new LMSDiscreteScheduler() This code sample shows the general process that takes place when the text and image embeddings are run through the U-Net and denoised by the scheduler. This denoising process is repeated for N number of steps. These new image embeddings are then used as the new input for the U-Net model. Using a scheduler algorithm, the output from the U-Net model is then used to compute new image embeddings. The U-Net model then reduces the noise (denoises) in the image using the text prompt as a conditional. ![]() The image and text embeddings are the initial input for the U-Net model. Var textPromptEmbeddings = TextProcessing.TextEncoder(textTokenized).ToArray() Denoise image loop Var textTokenized = TextProcessing.TokenizeText(prompt) var prompt = "a fireplace in an old cabin in the woods" This code sample shows the general process of tokenizing and encoding the input text prompt. The sample referenced in this post is inferencing only, so using the VAE is not required. Note that for training, you’ll also need to use the VAE to encode the images you use during training. ONNX Runtime Extensions is a library that extends the capability of the ONNX models and inference with ONNX Runtime by providing common pre and post-processing operators for vision, text, and NLP models. Instead of reimplementing it in C#, ONNX Runtime has created a cross-platform implementation using ONNX Runtime Extensions. OpenAI’s CLIP text tokenizer is written in Python. In the case of Stable Diffusion, the text and images are encoded into an embedding space that can be understood by the U-Net neural network as part of the denoising process. ![]() This numerical representation contains semantic meaning.
0 Comments
Leave a Reply. |