Marco does not know what the next big thing will be. He asked ChatGPT, though the answer was underwhelming.
But he’s sure that our future next-gen computing and networking infrastructure must be a viable platform for it.
Marco’s research spans a number of areas in computer systems, including distributed systems, large-scale/cloud
computing and computer networking with emphasis on programmable networks. His current focus is on designing better
systems support for AI/ML and providing practical implementations deployable in the real world.
Marco is a Professor of Computer Science at King Abdullah University of Science and Technology (KAUST).
Marco obtained his Ph.D. in computer science and engineering from the University of Genoa in 2009 after spending the
last year as a visiting student at the University of Cambridge.
He holds a laurea degree with honors in computer science and
engineering from the University of Genoa. He was a postdoctoral researcher at EPFL from 2009 to 2012 and after that
a senior research scientist for one year at Deutsche Telekom Innovation Labs & TU Berlin.
Before joining KAUST, he was an Assistant Professor in Computer Science at the Université catholique de Louvain.
He also held positions at Intel, Microsoft and Google.
Should SLMs be in the Driver's Seat? A Study of Local Models as Interpreters in Cloud-Written Agent Programs new
B. Radovič, M. Canini, V. Pejović
The 1st NeurIPS Workshop on SLMs for Agentic Systems (SLM-Agents'26), Dec 2026. To appear.
bibtex
@inproceedings{radovic2026slmdriver,
title={{Should SLMs be in the Driver's Seat? A Study of Local Models as Interpreters in Cloud-Written Agent Programs}},
author={Boris Radovič and Marco Canini and Veljko Pejović},
booktitle = {NeurIPS Workshop on SLMs for Agentic Systems (SLM-Agents)},
year={2026},
month={Dec}
}
No single Large Language Model (LLM) is uniformly reliable across queries, motivating multi-model inference systems that either route among models or combine their outputs. However, routing stops after selecting an initial model, while dense collaboration invokes peers on every query. We show that collaboration is non-monotonic: peers can recover failures that no model solves alone, but can also corrupt initially correct answers. We introduce CoMed (Controlled Model Escalation for Multi-LLM Deliberation), a post-anchor controller for selective cross-model collaboration. CoMed uses anchor self-consistency, router margin, and a lightweight peer probe to accept confident answers, verify ambiguous cases, and escalate only when collaboration is likely beneficial. We formalize this trade-off with a rescue–harm decomposition showing that selective collaboration improves when rescued errors outweigh collaboration-induced harms. Across medical, scientific, and general reasoning benchmarks, CoMed achieves higher mean accuracy than its corresponding fixed or routed anchor in all 16 open-weight settings, with gains up to +10.7 percentage points on MedQA while invoking fewer models and using fewer decoded tokens than dense collaboration. On Humanity’s Last Exam (HLE) with frontier models, CoMed improves GPT-5.5 from 23.1% to 28.1%, outperforming dense collaboration and achieving the best results.
bibtex
@inproceedings{alballa2026comed,
title={{COMED: The Missing Middle Between Routing and Collaboration in Multi-LLM Inference}},
author={Norah Alballa and Wenxuan Zhang and Salma Kharrat and Fares Fourati and Zafar Ayyub Qazi and Mohamed Elhoseiny and Marco Canini},
booktitle = {AACL-IJCNLP},
year={2026},
month={Nov}
}
DynSL: Resource-Adaptive Model Training across Heterogeneous Edge Devices new
B. Radovič, I. E. Akkus, M. Canini, R. Chen, A. Dethise, I. Rimac, V. Pejović
In Proceedings of SoCC'26, Nov 2026. To appear.
bibtex
@inproceedings{radovic2026dynsl,
title={{DynSL: Resource-Adaptive Model Training across Heterogeneous Edge Devices}},
author={Boris Radovič and Istemi Ekin Akkus and Marco Canini and Ruichuan Chen and Alice Dethise and Ivica Rimac and Veljko Pejović},
booktitle = {SoCC},
year={2026},
month={Nov}
}
fRAC: Remote Accelerator Calls on FPGAs new
Z. Yang, K. B. Iyer, Y. Chen, R. Shu, Z. István, M. Canini, S. A. Fahmy
In Proceedings of ACM SIGOPS ATC'26, Nov 2026. To appear.
@inproceedings{yang2026frac,
title={{fRAC: Remote Accelerator Calls on FPGAs}},
author={Ziyi Yang and Krishnan B. Iyer and Yixi Chen and Ran Shu and Zsolt István and Marco Canini and Suhaib A. Fahmy},
booktitle = {Proceedings of ACM SIGOPS ATC'26},
year={2026},
month={Nov}
}
When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs new
K. Wang, T. Lyu, G. Su, L. Yin, M. Canini, J. Geiping, S. Liu
Layer pruning has emerged as a widely adopted technique for improving the efficiency of large language models (LLMs). Although existing methods demonstrate strong performance retention on general knowledge tasks, their effect on long-chain reasoning, a more brittle yet crucial capability, remains largely unexplored. In this work, we study the impact of layer pruning on long-chain reasoning through the lens of test-time scaling, a key mechanism in modern LLMs that enables strong reasoning capacity by allocating more computation at inference time. With extensive experiments, we demonstrate that pruning even one or two layers can severely impair test-time scaling, with performance collapsing drastically on long reasoning benchmarks even when performance on knowledge-intensive and shallow reasoning tasks remains stable. Furthermore, we find that standard supervised fine-tuning remedies fail to recover test-time scaling once it has deteriorated. Through in-depth analyses, we identify the mechanisms underlying this fragility of test-time scaling and highlight the fundamental risks of applying layer pruning to reasoning-intensive LLMs. These findings call for a rethinking of layer pruning strategies and provide insights for developing methods that preserve the robustness of reasoning.
bibtex
@inproceedings{wang2026fewerlayersbreakchains,
title={{When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs}},
author={Keyu Wang and Tian Lyu and Guinan Su and Lu Yin and Marco Canini and Jonas Geiping and Shiwei Liu},
booktitle={Proceedings of COLM'26},
year={2026},
month={Oct}
}
Agentic LLM workflows issue many dependent calls with unpredictable resource demand, causing queue buildup and latency degradation on shared serving backends when left unmanaged. In this paper, we propose CALM-MAS, a congestion-aware serving framework for LLM applications that treats LLM test-time computation as an elastic resource, dynamically adjusting the compute profile of admitted tasks to tame congestion. CALM-MAS detects early signals of backend saturation, and leverages the flexibility of LLM applications to regulate load. During spikes of requests, the system downgrades agent topology and reasoning depth; during low-utilization periods, it allocates additional reasoning effort to maximize task accuracy. Compared with a static serving baseline based on vLLM, CALM-MAS reduces shared-backend tail latency by 77% with the accuracy degradation remaining confined to 6.1 pps relative to the native agent configuration.
bibtex
@inproceedings{bennasr2026calmmas,
title={{Congestion-Aware Serving of Agentic LLM Applications}},
author={Mouheb Ben Nasr and Muhammad Bilal and Alessandro Cornacchia and Boris Radovič and Marco Canini},
booktitle = {APSys},
year={2026},
month={Sep}
}
Embodied LLM systems increasingly co-locate latency-critical robotics pipelines with compute- and memory-intensive language-model inference on edge platforms such as NVIDIA Jetson. This co-location avoids cloud round trips and enables privacy-preserving, low-latency interaction, but it also creates a new source of resource interference. Our experiments show that the deadline miss rate of a real-time task can reach nearly 100% under co-running LLM inference. The root cause is that, although robotics workloads and LLM inference typically execute on the CPU and GPU, respectively, they share the same unified-memory subsystem on edge platforms. As a result, Memory-Controller (MC) capacity can become a hidden bottleneck even when conventional compute resources do not appear saturated. Based on this finding, we propose MCSched, a lightweight MC-aware runtime scheduler for embodied LLM systems. MCSched combines coarse MC-pressure signals with foreground timing feedback and temporarily gates background LLM execution when robotics tasks approach deadline risk. In our prototype experiments, MCSched successfully reduced the deadline miss rate of the camera task and PCL task by up to 83.35% and 74.11%, respectively, on the ROS-LLM framework while maintaining LLM inference speed. These observations provide preliminary but compelling evidence that MC-aware scheduling is a practical operating system/runtime direction for deployable embodied AI on unified memory edge devices.
bibtex
@inproceedings{mei2026mcsched,
title={{MCSched: Memory-Controller-Aware Scheduling for Embodied LLM Workloads on NVIDIA Jetson}},
author={Teng Mei and Chengxuan Pei and Marco Canini and Yunming Xiao},
booktitle = {APSys},
year={2026},
month={Sep}
}
