In many cases, yes. Azure Advisor analyzes actual resource utilization and recommends right-sizing (or shutting down idle resources), and many VM resizing operations can be done with only a brief restart rather than a full rebuild — though a short restart is still typically required for most VM size changes.
For workloads where even brief downtime is unacceptable, techniques like deploying a right-sized replacement alongside the original and cutting over can minimize disruption further, though this adds complexity. A partner experienced in Azure cost optimization can help plan the approach that fits your tolerance for disruption.