Marco does not know what the next big thing will be. He asked ChatGPT, though the answer was underwhelming.
But he’s sure that our future next-gen computing and networking infrastructure must be a viable platform for it.
Marco’s research spans a number of areas in computer systems, including distributed systems, large-scale/cloud
computing and computer networking with emphasis on programmable networks. His current focus is on designing better
systems support for AI/ML and providing practical implementations deployable in the real world.
Marco is a Professor of Computer Science at King Abdullah University of Science and Technology (KAUST).
Marco obtained his Ph.D. in computer science and engineering from the University of Genoa in 2009 after spending the
last year as a visiting student at the University of Cambridge.
He holds a laurea degree with honors in computer science and
engineering from the University of Genoa. He was a postdoctoral researcher at EPFL from 2009 to 2012 and after that
a senior research scientist for one year at Deutsche Telekom Innovation Labs & TU Berlin.
Before joining KAUST, he was an Assistant Professor in Computer Science at the Université catholique de Louvain.
He also held positions at Intel, Microsoft and Google.
Should SLMs be in the Driver's Seat? A Study of Local Models as Interpreters in Cloud-Written Agent Programs new
B. Radovič, M. Canini, V. Pejović
The 1st NeurIPS Workshop on SLMs for Agentic Systems (SLM-Agents'26), Dec 2026. To appear.
abstract
The most accurate agents executing user-provided tasks are driven by large, cloud-hosted models, which solve them by reading the private data of the user. Agents powered by models small enough to run on edge devices keep that data local, but yield far worse results. A third option is to have cloud models write a program that devices execute: the data stays local, but as the program is written before any data is seen, it fails whenever a step depends on interpreting content that appears only at run time. We explore an intermediate architecture, in which the cloud-written program defers bounded questions over user data to a small local model through an ordinary function call. On AppWorld this improves the success rate of a code-only agent in all six combinations of backbone model and test partition we evaluate, by up to 14.9pp, recovering a substantial part of the gap to a cloud-hosted ReAct loop while keeping user data on the device.
bibtex
@inproceedings{radovic2026should,
author = {Radovič, Boris and Canini, Marco and Pejović, Veljko},
title = {{Should SLMs be in the Driver's Seat? A Study of Local Models as Interpreters in Cloud-Written Agent Programs}},
booktitle = {NeurIPS Workshop on SLMs for Agentic Systems},
series = {SLM-Agents '26},
year = {2026},
month = {Dec}
}
No single Large Language Model (LLM) is uniformly reliable across queries, motivating multi-model inference systems that either route among models or combine their outputs. However, routing stops after selecting an initial model, while dense collaboration invokes peers on every query. We show that collaboration is non-monotonic: peers can recover failures that no model solves alone, but can also corrupt initially correct answers. We introduce CoMed (Controlled Model Escalation for Multi-LLM Deliberation), a post-anchor controller for selective cross-model collaboration. CoMed uses anchor self-consistency, router margin, and a lightweight peer probe to accept confident answers, verify ambiguous cases, and escalate only when collaboration is likely beneficial. We formalize this trade-off with a rescue–harm decomposition showing that selective collaboration improves when rescued errors outweigh collaboration-induced harms. Across medical, scientific, and general reasoning benchmarks, CoMed achieves higher mean accuracy than its corresponding fixed or routed anchor in all 16 open-weight settings, with gains up to +10.7 percentage points on MedQA while invoking fewer models and using fewer decoded tokens than dense collaboration. On Humanity’s Last Exam (HLE) with frontier models, CoMed improves GPT-5.5 from 23.1% to 28.1%, outperforming dense collaboration and achieving the best results.
bibtex
@inproceedings{alballa2026comed,
author = {Alballa, Norah and Zhang, Wenxuan and Kharrat, Salma and Fourati, Fares and Qazi, Zafar Ayyub and Elhoseiny, Mohamed and Canini, Marco},
title = {{COMED: The Missing Middle Between Routing and Collaboration in Multi-LLM Inference}},
booktitle = {Proceedings of the Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and International Joint Conference on Natural Language Processing},
series = {AACL-IJCNLP '26},
year = {2026},
month = {Nov},
publisher = {Association for Computational Linguistics}
}
Training models over data distributed across edge devices presents a trade-off: centralizing data simplifies training but raises privacy and regulatory concerns, while full local training may exceed the devices’ compute and memory capacity. Split learning addresses this challenge by partitioning the model between clients and the server, allowing part of the computation to be offloaded. However, existing approaches typically rely on static split configurations or simplified assumptions that fail to capture the heterogeneity and complex dynamics of real deployments. We present DynSL, a framework that dynamically allocates model layers between clients and the server by continuously monitoring system metrics and periodically solving an optimization problem that accounts for client and server computation and communication. When beneficial, the system reconfigures the split, enabling the offloading strategy to adapt to changing device capabilities and system conditions. Extensive experiments on a testbed of 24 heterogeneous edge devices show that DynSL reduces training time by up to ∼30% over the strongest adaptive baseline and by more than 50% over a suboptimal uniform split, with no loss – and sometimes a gain – in accuracy.
bibtex
@inproceedings{radovic2026dynsl,
author = {Radovič, Boris and Akkus, Istemi Ekin and Canini, Marco and Chen, Ruichuan and Dethise, Alice and Rimac, Ivica and Pejović, Veljko},
title = {{DynSL: Resource-Adaptive Model Training across Heterogeneous Edge Devices}},
booktitle = {Proceedings of the ACM Symposium on Cloud Computing},
series = {SoCC '26},
year = {2026},
month = {Nov},
publisher = {ACM},
doi = {10.1145/3828161.3857850},
url = {https://doi.org/10.1145/3828161.3857850}
}
Hardware accelerators are essential for meeting microsecond-scale demands in distributed applications, but invoking them over the network remains bottlenecked by host software. Host-mediated paths (RPC, RDMA) still rely on the CPU to reassemble requests and launch accelerators, which dominates end-to-end latency, while packet-level offload on SmartNICs lacks any notion of a complete request. We introduce the Remote Accelerator Call (RAC) paradigm, in which transport ingestion, request reassembly, dispatch, and accelerator invocation are handled entirely in hardware. We build fRAC, a hostless prototype that implements this network-to-accelerator interface as a lightweight shim on an FPGA hosting multiple virtualized accelerators, reconfigurable at runtime in a few milliseconds. Serverless functions backed by fine-grained accelerators are a natural fit, since host-mediated invocation overhead can match or exceed their short execution times. fRAC achieves under 7.8 μs end-to-end latency while sustaining application throughput of up to 90 Gbps. Compared to highly optimized DPDK-based CPU and DPU baselines, it achieves at least 80% lower latency under saturation, while consuming 85% less power.
bibtex
@inproceedings{yang2026frac,
author = {Yang, Ziyi and Iyer, Krishnan B. and Chen, Yixi and Shu, Ran and István, Zsolt and Canini, Marco and Fahmy, Suhaib A.},
title = {{fRAC: Remote Accelerator Calls on Hostless FPGAs}},
booktitle = {Proceedings of the ACM SIGOPS Annual Technical Conference},
series = {SIGOPS-ATC '26},
year = {2026},
month = {Nov},
publisher = {ACM},
doi = {10.1145/3828523.3857070},
url = {https://doi.org/10.1145/3828523.3857070}
}
@inproceedings{pasqualoni2026ccwse3,
author = {Pasqualoni, Saverio and Canini, Marco and De Sensi, Daniele},
title = {{Exploring Tree-Based Approaches for Collective Communications on Cerebras WSE-3}},
booktitle = {Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis},
series = {SC '26},
year = {2026},
month = {Nov},
publisher = {ACM}
}
When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs new
K. Wang, T. Lyu, G. Su, L. Yin, M. Canini, J. Geiping, S. Liu
Layer pruning has emerged as a widely adopted technique for improving the efficiency of large language models (LLMs). Although existing methods demonstrate strong performance retention on general knowledge tasks, their effect on long-chain reasoning, a more brittle yet crucial capability, remains largely unexplored. In this work, we study the impact of layer pruning on long-chain reasoning through the lens of test-time scaling, a key mechanism in modern LLMs that enables strong reasoning capacity by allocating more computation at inference time. With extensive experiments, we demonstrate that pruning even one or two layers can severely impair test-time scaling, with performance collapsing drastically on long reasoning benchmarks even when performance on knowledge-intensive and shallow reasoning tasks remains stable. Furthermore, we find that standard supervised fine-tuning remedies fail to recover test-time scaling once it has deteriorated. Through in-depth analyses, we identify the mechanisms underlying this fragility of test-time scaling and highlight the fundamental risks of applying layer pruning to reasoning-intensive LLMs. These findings call for a rethinking of layer pruning strategies and provide insights for developing methods that preserve the robustness of reasoning.
bibtex
@inproceedings{wang2026when,
author = {Wang, Keyu and Lyu, Tian and Su, Guinan and Yin, Lu and Canini, Marco and Geiping, Jonas and Liu, Shiwei},
title = {{When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs}},
booktitle = {Proceedings of the Conference on Language Modeling},
series = {COLM '26},
year = {2026},
month = {Oct}
}
