FreeSBC VM High CPU usage and Scheduling problem alarm

[stringpark]

Hello,

I am testing FreeSBC in a homelab, non-production environment. Once in use, it will only process one or two calls at a time. I have the KVM image loaded up on a Proxmox VM, which was created with the processor type set to "host", allocated all 4 available cores (Proxmox is running on a laptop), 4GB of memory, and two virtual network interfaces (one for mgmt0 and one for voip0). All appears to be up and running and I have thus far only run through the short initial setup wizard and loaded the license. However, I have to immediate concerns: 1.) The VM consistently runs at ~30% CPU usage and "top" shows "tbrouter" running consistently at ~100% CPU. This is even though FreeSBC hasn't been configured to process any traffic yet and is basically sitting there idle.  2.) Under Security>SBC>Status, "Scheduling problem alarm" shows "true". I found the Troubleshooting Guide that addresses this alarm and it lists the below as possible causes, but unfortunately I don't think any of these are the issue in my case:

• Non-dedicated (or not ‘pinned’) CPU [Open-stack] > I don't use Open-stack, but did check the "Dedicated CPU" box for the voip0 interface during the initial setup wizard. Not sure if that is what is being referred to here or not.
  
• Not enough CPU reservation [Vmware] > The VM was given all cores and CPU limit set to "unlimited". Although I do have other VMs running on the machine, they are pulling virtually no CPU resources.
  
• Memory is not dedicated to the Virtual Machine (VM) > The VM was given 4GB memory with "ballooning" turned off.
  
• Too many active virtual machines fighting for resources on the host > Other VMs are running on the machine, but using very little resources. Ample resources are available to satisfy the allotment to the FreeSBC VM.
  
• On KVM based installations (proxmox,virtmanager) CPU type must set to “host” > CPU type is set to "host".
  

Any advice would be welcome.

Thank you!

[jpuckett]

Hi Stringpack,

1.) The VM consistently runs at ~30% CPU usage and "top" shows "tbrouter" running consistently at ~100% CPU.

This is a reservation on one of your Cores for tbrouter. Nothing to be concerned about, if you do a

Code:

Shift + I

while viewing top it will change the Irix mode, and display total value of percentage instead of a single core value. If you do that you should see tbrouter drop down to about 25% (One fully reserved core of the four you have available).

 2.) Under Security>SBC>Status, "Scheduling problem alarm" shows "true".

Proxmox isn't officially supported but we have customers using it in the field successfully and I use it for my SBC in my home lab as well without issues so I think we can resolve this.

Could you tell me what CPU this laptop is using? Our most intensive applications (like tbrouter) are using a single thread, so it has decreased efficiency on consumer chips(compared to server), it's possible the one core reserved for tbrouter is being split with the host and even though there is not any serious call traffic, tbrouter can't get the heartbeats it needs out in an acceptable time. Thus throwing the warning.

Could you also check for me if you have Non-uniform Memory Access (NUMA) enabled on the guest machine? This technology tries to avoid bottlenecks by swapping around reservations and can cause our applications to freak out when the switch happens.


Let me know the answers to those two questions and we'll find a way forward.

Cheers,
Joshua
TB Support

[stringpark]

@jpuckett,

Thanks so much for the reply, I really appreciate your assistance with this! Yes, I do see now exactly what you described with the Shift + I command using top. It shows ~25% for tbrouter as you predicted.

Concerning the processor model, it is an i7-3540M. Not a lot of horsepower there, but I was hoping it would be sufficient for minimal testing with FreeSBC and FusionPBX. "Enable NUMA" is deselected in the VM configuration.

Thanks!

[jpuckett]

Hey Stringpark,

I think that CPU is good enough for the tests you need, can you generate a tbreport for a few days and post an upload link?
Or send it to support@telcobridges.com referencing this forum post.

I'd like to see what application is complaining about scheduling and how long of a gap it is for.

Cheers,
Joshua