## Batch Size 128 Compile true

| Experiment | Warmup_latency (s) | Average_latency (s) | Throughput (samples/sec) | GPU Utilization (%) |
| ---------- | ------------------ | ------------------- | ------------------------ | ------------------- |
| original | 14.358 +/- 0.981 | 14.250 +/- 0.758 | 522.998 +/- 20.830 | 55.501 +/- 2.123 |
| h2d_d2h_threads | 12.520 +/- 0.253 | 12.774 +/- 0.714 | 600.578 +/- 27.662 | 61.534 +/- 3.653 |