Marco does not know what the next big thing will be. He asked ChatGPT, though the answer was underwhelming.
But he’s sure that our future next-gen computing and networking infrastructure must be a viable platform for it.
Marco’s research spans a number of areas in computer systems, including distributed systems, large-scale/cloud
computing and computer networking with emphasis on programmable networks. His current focus is on designing better
systems support for AI/ML and providing practical implementations deployable in the real world.
Marco is a Professor of Computer Science at King Abdullah University of Science and Technology (KAUST).
Marco obtained his Ph.D. in computer science and engineering from the University of Genoa in 2009 after spending the
last year as a visiting student at the University of Cambridge.
He holds a laurea degree with honors in computer science and
engineering from the University of Genoa. He was a postdoctoral researcher at EPFL from 2009 to 2012 and after that
a senior research scientist for one year at Deutsche Telekom Innovation Labs & TU Berlin.
Before joining KAUST, he was an Assistant Professor in Computer Science at the Université catholique de Louvain.
He also held positions at Intel, Microsoft and Google.
Should SLMs be in the Driver's Seat? A Study of Local Models as Interpreters in Cloud-Written Agent Programs new
B. Radovič, M. Canini, V. Pejović
The 1st NeurIPS Workshop on SLMs for Agentic Systems (SLM-Agents'26), Dec 2026. To appear.
abstract
The most accurate agents executing user-provided tasks are driven by large, cloud-hosted models, which solve them by reading the private data of the user. Agents powered by models small enough to run on edge devices keep that data local, but yield far worse results. A third option is to have cloud models write a program that devices execute: the data stays local, but as the program is written before any data is seen, it fails whenever a step depends on interpreting content that appears only at run time. We explore an intermediate architecture, in which the cloud-written program defers bounded questions over user data to a small local model through an ordinary function call. On AppWorld this improves the success rate of a code-only agent in all six combinations of backbone model and test partition we evaluate, by up to 14.9pp, recovering a substantial part of the gap to a cloud-hosted ReAct loop while keeping user data on the device.
bibtex
@inproceedings{radovic2026should,
author = {Radovič, Boris and Canini, Marco and Pejović, Veljko},
title = {{Should SLMs be in the Driver's Seat? A Study of Local Models as Interpreters in Cloud-Written Agent Programs}},
booktitle = {NeurIPS Workshop on SLMs for Agentic Systems},
series = {SLM-Agents '26},
year = {2026},
month = {Dec}
}
No single Large Language Model (LLM) is uniformly reliable across queries, motivating multi-model inference systems that either route among models or combine their outputs. However, routing stops after selecting an initial model, while dense collaboration invokes peers on every query. We show that collaboration is non-monotonic: peers can recover failures that no model solves alone, but can also corrupt initially correct answers. We introduce CoMed (Controlled Model Escalation for Multi-LLM Deliberation), a post-anchor controller for selective cross-model collaboration. CoMed uses anchor self-consistency, router margin, and a lightweight peer probe to accept confident answers, verify ambiguous cases, and escalate only when collaboration is likely beneficial. We formalize this trade-off with a rescue–harm decomposition showing that selective collaboration improves when rescued errors outweigh collaboration-induced harms. Across medical, scientific, and general reasoning benchmarks, CoMed achieves higher mean accuracy than its corresponding fixed or routed anchor in all 16 open-weight settings, with gains up to +10.7 percentage points on MedQA while invoking fewer models and using fewer decoded tokens than dense collaboration. On Humanity’s Last Exam (HLE) with frontier models, CoMed improves GPT-5.5 from 23.1% to 28.1%, outperforming dense collaboration and achieving the best results.
bibtex
@inproceedings{alballa2026comed,
author = {Alballa, Norah and Zhang, Wenxuan and Kharrat, Salma and Fourati, Fares and Qazi, Zafar Ayyub and Elhoseiny, Mohamed and Canini, Marco},
title = {{COMED: The Missing Middle Between Routing and Collaboration in Multi-LLM Inference}},
booktitle = {Proceedings of the Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and International Joint Conference on Natural Language Processing},
series = {AACL-IJCNLP '26},
year = {2026},
month = {Nov},
publisher = {Association for Computational Linguistics}
}
DynSL: Resource-Adaptive Model Training across Heterogeneous Edge Devices new
B. Radovič, I. E. Akkus, M. Canini, R. Chen, A. Dethise, I. Rimac, V. Pejović
SoCC'26, Nov 2026. To appear.
bibtex
@inproceedings{radovic2026dynsl,
author = {Radovič, Boris and Akkus, Istemi Ekin and Canini, Marco and Chen, Ruichuan and Dethise, Alice and Rimac, Ivica and Pejović, Veljko},
title = {{DynSL: Resource-Adaptive Model Training across Heterogeneous Edge Devices}},
booktitle = {Proceedings of the ACM Symposium on Cloud Computing},
series = {SoCC '26},
year = {2026},
month = {Nov},
publisher = {ACM}
}
fRAC: Remote Accelerator Calls on Hostless FPGAs new
Z. Yang, K. B. Iyer, Y. Chen, R. Shu, Z. István, M. Canini, S. A. Fahmy
@inproceedings{yang2026frac,
author = {Yang, Ziyi and Iyer, Krishnan B. and Chen, Yixi and Shu, Ran and István, Zsolt and Canini, Marco and Fahmy, Suhaib A.},
title = {{fRAC: Remote Accelerator Calls on Hostless FPGAs}},
booktitle = {Proceedings of the ACM SIGOPS Annual Technical Conference},
series = {SIGOPS-ATC '26},
year = {2026},
month = {Nov},
publisher = {ACM}
}
@inproceedings{pasqualoni2026ccwse3,
author = {Pasqualoni, Saverio and Canini, Marco and De Sensi, Daniele},
title = {{Exploring Tree-Based Approaches for Collective Communications on Cerebras WSE-3}},
booktitle = {Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis},
series = {SC '26},
year = {2026},
month = {Nov},
publisher = {ACM}
}
When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs new
K. Wang, T. Lyu, G. Su, L. Yin, M. Canini, J. Geiping, S. Liu
Layer pruning has emerged as a widely adopted technique for improving the efficiency of large language models (LLMs). Although existing methods demonstrate strong performance retention on general knowledge tasks, their effect on long-chain reasoning, a more brittle yet crucial capability, remains largely unexplored. In this work, we study the impact of layer pruning on long-chain reasoning through the lens of test-time scaling, a key mechanism in modern LLMs that enables strong reasoning capacity by allocating more computation at inference time. With extensive experiments, we demonstrate that pruning even one or two layers can severely impair test-time scaling, with performance collapsing drastically on long reasoning benchmarks even when performance on knowledge-intensive and shallow reasoning tasks remains stable. Furthermore, we find that standard supervised fine-tuning remedies fail to recover test-time scaling once it has deteriorated. Through in-depth analyses, we identify the mechanisms underlying this fragility of test-time scaling and highlight the fundamental risks of applying layer pruning to reasoning-intensive LLMs. These findings call for a rethinking of layer pruning strategies and provide insights for developing methods that preserve the robustness of reasoning.
bibtex
@inproceedings{wang2026when,
author = {Wang, Keyu and Lyu, Tian and Su, Guinan and Yin, Lu and Canini, Marco and Geiping, Jonas and Liu, Shiwei},
title = {{When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs}},
booktitle = {Proceedings of the Conference on Language Modeling},
series = {COLM '26},
year = {2026},
month = {Oct}
}
Agentic LLM workflows issue many dependent calls with unpredictable resource demand, causing queue buildup and latency degradation on shared serving backends when left unmanaged. In this paper, we propose CALM-MAS, a congestion-aware serving framework for LLM applications that treats LLM test-time computation as an elastic resource, dynamically adjusting the compute profile of admitted tasks to tame congestion. CALM-MAS detects early signals of backend saturation, and leverages the flexibility of LLM applications to regulate load. During spikes of requests, the system downgrades agent topology and reasoning depth; during low-utilization periods, it allocates additional reasoning effort to maximize task accuracy. Compared with a static serving baseline based on vLLM, CALM-MAS reduces shared-backend tail latency by 77% with the accuracy degradation remaining confined to 6.1 pps relative to the native agent configuration.
bibtex
@inproceedings{bennasr2026calmmas,
author = {Ben Nasr, Mouheb and Bilal, Muhammad and Cornacchia, Alessandro and Radovič, Boris and Canini, Marco},
title = {{Congestion-Aware Serving of Agentic LLM Applications}},
booktitle = {Proceedings of the ACM SIGOPS Asia-Pacific Workshop on Systems},
series = {APSys '26},
year = {2026},
month = {Sep},
publisher = {ACM}
}
Embodied LLM systems increasingly co-locate latency-critical robotics pipelines with compute- and memory-intensive language-model inference on edge platforms such as NVIDIA Jetson. This co-location avoids cloud round trips and enables privacy-preserving, low-latency interaction, but it also creates a new source of resource interference. Our experiments show that the deadline miss rate of a real-time task can reach nearly 100% under co-running LLM inference. The root cause is that, although robotics workloads and LLM inference typically execute on the CPU and GPU, respectively, they share the same unified-memory subsystem on edge platforms. As a result, Memory-Controller (MC) capacity can become a hidden bottleneck even when conventional compute resources do not appear saturated. Based on this finding, we propose MCSched, a lightweight MC-aware runtime scheduler for embodied LLM systems. MCSched combines coarse MC-pressure signals with foreground timing feedback and temporarily gates background LLM execution when robotics tasks approach deadline risk. In our prototype experiments, MCSched successfully reduced the deadline miss rate of the camera task and PCL task by up to 83.35% and 74.11%, respectively, on the ROS-LLM framework while maintaining LLM inference speed. These observations provide preliminary but compelling evidence that MC-aware scheduling is a practical operating system/runtime direction for deployable embodied AI on unified memory edge devices.
bibtex
@inproceedings{mei2026mcsched,
author = {Mei, Teng and Pei, Chengxuan and Canini, Marco and Xiao, Yunming},
title = {{MCSched: Memory-Controller-Aware Scheduling for Embodied LLM Workloads on NVIDIA Jetson}},
booktitle = {Proceedings of the ACM SIGOPS Asia-Pacific Workshop on Systems},
series = {APSys '26},
year = {2026},
month = {Sep},
publisher = {ACM}
}
