Frequently Asked Questions (FAQ) ================================ This page answers the most common questions about using **GATI** and **GATICC**. Why does GATICC require 5D input tensors? ----------------------------------------- GATI’s internal dataflow expects input in the format: (N, 1, C, H, W) This extra dimension (the second ``1``) is used internally by the hardware to manage memory alignment and accelerator pipeline scheduling. Even if your ONNX model uses ``(N, C, H, W)``, you must reshape your input before calling ``gati.run()`` or ``gati.sim()``: inp = inp.reshape(1, 1, C, H, W) This requirement is the same for classification and object detection models. What do the numbers in gati.set_arch() mean? -------------------------------------------- The call: gati.set_arch(ramsize=512, sa_arch="9,4,4", vasize=32, accbuf_size=4096, fcbuf_size=32768, im2colbuf_size=1024) configures the hardware runtime. Here is what each parameter means: - **ramsize** Total DRAM (in MB) allocated for the FPGA. - **sa_arch** hardware configuration. Examples: - ``9,4,4`` - ``9,8,8`` - ``16,1,16`` These numbers describe the compute tile structure inside the FPGA. Different configurations work better for different models. - **vasize** DRAM bandwidth in bytes per cycle. - **accbuf_size** Accumulator buffer size used during convolution accumulation. - **fcbuf_size** Buffer used for fully-connected layers. - **im2colbuf_size** Buffer size used for im2col operations in convolution lowering. These defaults are correct for most models, but advanced users may tune them. For the full list of architecture flags and descriptions: gaticc --help Why does GATI use only ONNX format? ----------------------------------- ONNX is: - open standard - framework-independent - optimized for deployment - widely supported by quantization tools It allows users from PyTorch, TensorFlow, Keras, MXNet, etc. to export models easily. GATICC currently supports **INT8 ONNX** for FPGA and **FP32/INT8 ONNX** for simulation. In future updates, more formats may be supported. What is a .gml file? --------------------- A ``.gml`` file is the **compiled model format** used by GATI. When you run: gati.compile("model_int8.onnx", "model.gml") GATICC converts your ONNX model into a hardware-friendly representation called **GATI Model Language (GML)**. In simple terms: **ONNX → GML → FPGA Execution** Why do we need GML? ------------------- Because ONNX is a *high-level* framework-agnostic format. It is not optimized for direct FPGA execution. GML is: - compact - hardware-aligned - ready for the GATI runtime to execute This allows very fast, low-latency inference on the Vaaman FPGA. Where is the .gml file used? ---------------------------- The GML file is loaded on the FPGA using: gati.load("model_int8.onnx", "model.gml") After loading, you can repeatedly call: gati.run({name: input_tensor}) without needing to recompile or reload the model. Why does FPGA inference require INT8 quantized models? ------------------------------------------------------ FPGA hardware has been optimized for **INT8 arithmetic** because: - it provides best speed and lowest power - reduces memory usage - simplifies hardware design - is used by nearly all edge inference accelerators Floating-point inference (FP32) is supported only in **simulation mode**. What operators are supported on FPGA? ------------------------------------- GATI currently supports the following INT8 operators: - QLinearConv - standard convolution - depthwise convolution - pointwise convolution - QGemm - Relu, Clip - QLinearAdd, QLinearSub, QLinearMul - MaxPool, AveragePool - Flatten More operators will be added in future releases. Can I run FP32 models on the FPGA? ---------------------------------- No — FPGA execution supports **INT8 only**. However: - FP32 models **CAN** run using ``gati.sim()`` - You can compare FP32 simulation results with INT8 FPGA output - This helps debugging quantization issues What is the difference between Load and Run? -------------------------------------------- - ``gati.load(onnx_path, gml_path)`` Loads the compiled model onto the FPGA and initializes the runtime. - ``gati.run({name: tensor})`` Runs inference on the model already loaded. You must call **load()** once with the required arguments before using **run()** for inference. Why doesn’t GATICC perform preprocessing or postprocessing automatically? ------------------------------------------------------------------------- Because: - Each ONNX model uses different preprocessing - Formats vary widely (RGB/BGR, normalization, resizing, etc.) - Object detection models require custom decoding, anchors, NMS - Many models require application-specific logic Therefore, GATICC only accepts **numpy tensors** that match model input shape. Users must write preprocessing and postprocessing functions depending on their model. How can I see all available commands and flags? ----------------------------------------------- Run: gaticc --help This shows every compiler flag, runtime option, debug option, and arch setting. Will more operators and models be supported in the future? ---------------------------------------------------------- Yes. Upcoming updates may include: - more convolution variants - additional quantized operators - better support for detection/segmentation models - extended Python APIs - more advanced bitstreams Roadmap will be updated along with each release. Does GATICC support multiple outputs? ----------------------------------------------- Yes. ``gati.run()`` returns a list of: [("tensor_name", numpy_array), ...] What should I do if I get “shape mismatch” or “input missing” errors? --------------------------------------------------------------------- Check the following: 1. Your tensor is 5D: ``(N, 1, C, H, W)`` 2. ``input_name`` matches exactly what ``gati.get_model_inputs()`` returns 3. dtype is correct 4. Preprocessing is correct for your model If issues remain, test the model first using: gati.sim("model.onnx", {name: tensor}) Can I run multiple images (batching)? ------------------------------------- Yes. GATICC fully supports batching as long as the input tensor follows the required 5D format: (N, 1, C, H, W) Where: - ``N`` = batch size (number of images) - ``1`` = internal dimension required by GATI - ``C, H, W`` = channels, height, width of each image Example: passing 8 images at once inp = np.stack([img1, img2, ..., img8], axis=0) # shape: (8, 1, C, H, W) out = gati.run({name: inp}) Batching reduces overhead and improves throughput for many models. Can I use camera input directly? -------------------------------- Yes. Camera input is handled as part of your **preprocessing** code. You can: 1. capture frames using OpenCV, 2. preprocess each frame into a 5D tensor, 3. send it to ``gati.run()`` inside a loop. Example: cap = cv2.VideoCapture(0) while True: ret, frame = cap.read() if not ret: break inp = preprocess(frame) # convert to (1,1,C,H,W) out = gati.run({name: inp}) # inference # postprocess + display... GATICC only requires the final input tensor. Capturing, resizing, and formatting camera frames is the user's responsibility. Complete camera-based examples are included in the ``examples/`` directory.