@HenrikEnquist I can't reproduce the volume bump anymore so it might be fixed in alpha 2. I connect my XMOS usb interface to my motu mk5 with toslink, set the clock of my motu mk5 to external(so its clock follows the upstream XMOS interface), and lastly motu mk5 is connected to Pi running CDSP with a USB cable. When I use ASIO driver of XMOS on windows to play music (eg. play music with ASIO plugin in foobar), there're lots of buffer underrun warning in CDSP's log, especially if I stop the music and replay it after a while. By directplay, I mean play music without ASIO driver(in a browser), so basically windows directstream. I can't really notice any actual underrun(clicks or popping sound) though. Chunksize is 1024 at 48kHz. I also tried 2048 for chunksize. It makes no difference. Underrun still shows up in log from time to time, and I still won't be cable to catch any click or pops. So I guess it could be XMOS ASIO driver cut itself off when not playing any music.
@HenrikEnquist I'm seeing much higher CPU usage on alpha 3 than v1.0.3. Is this expected?
On v1.0.3 I get total CPU usage for the CamillaDSP process of 22% with the top thread using 12%. On alpha 3 I get total CPU usage of 95% with the top thread using 85% - I assume that's the processing thread.
My use case is I'm upsampling to 352kHz before the audio gets to CamillaDSP, then using CamillaDSP for headphone crossfeed and PEQ - so the pipeline consists almost entirely of a series of biquad filters. The CPU usage drops significantly if I reduce the number of biquads so it seems for some reason biquads are very CPU heavy in alpha 3. I'm running this on an Intel N3700 quad core atom CPU.
Thank you for your great work!
On v1.0.3 I get total CPU usage for the CamillaDSP process of 22% with the top thread using 12%. On alpha 3 I get total CPU usage of 95% with the top thread using 85% - I assume that's the processing thread.
My use case is I'm upsampling to 352kHz before the audio gets to CamillaDSP, then using CamillaDSP for headphone crossfeed and PEQ - so the pipeline consists almost entirely of a series of biquad filters. The CPU usage drops significantly if I reduce the number of biquads so it seems for some reason biquads are very CPU heavy in alpha 3. I'm running this on an Intel N3700 quad core atom CPU.
Thank you for your great work!
No, there hasn't been any changes in the dsp code. Could you share your full configs for both v1 and v2?
If that is the case then it would explain the difference. @spfenwick are you running the binaries from GitHub?
I just tried this config with the binaries from github. I don't see any difference between 1.0.3 and 2.0.0-a3.
I started off using the linux-amd64 binary from github and that gave me the high CPU load.
I've also tried compiling from source:
- Using the default settings gives high CPU load.
- Compiling with "RUSTFLAGS='-C target-cpu=x86-64' cargo build --release" gives high CPU load. I think that's the same as default.
- Compiling with "RUSTFLAGS='-C target-cpu=x86-64-v2' cargo build --release" gives CPU load more in line with v1.0.3
- Compiling with "RUSTFLAGS='-C target-cpu=x86-64-v3' cargo build --release" gives a binary that crashes on my PC
So target-cpu=x86-64-v2 seems to give a solution for my purposes, though presumably that makes the binary compatible with fewer old CPUs.
No they use the same settings, so no sse4.2 on either. It might me some regression in the rustc compiler. It could be worth trying a couple of older versions to check that.
I'm seeing some weird results that make me think this is a pre-existing bug that is now being triggered in 2.0.0a3. I don't think it's related to sse4.2, though enabling sse4.2 seems to mask it for some reason.
This should be easy to reproduce. First generate a 16-bit stereo raw file, say with sox:
The reason I think this is a pre-existing bug is if I remove the Gain filter from the start of the pipeline I get similar results with v1.0.3.
The modified config file is:
Both the binaries I'm using are straight off github.
This should be easy to reproduce. First generate a 16-bit stereo raw file, say with sox:
Use this config file:
Time CamillaDSP v1.0.3 processing the file:
Doing the same thing with v2.0.0a3 is tricker as it goes into a loop at the end of the file rather than terminating so I need to quickly hit Ctrl-C when it starts looping:
The warnings are new and the CPU used is significantly higher - 11 seconds vs 1.8.
The reason I think this is a pre-existing bug is if I remove the Gain filter from the start of the pipeline I get similar results with v1.0.3.
The modified config file is:
And the results with v1.0.3:
i.e. the same warnings that v2.0.0a3 gave and similarly high CPU usage. I have no idea why just removing a filter from the start of the pipeline would have this effect.
Both the binaries I'm using are straight off github.
I'm trying to reproduce this but haven't had any luck so far. Unfortunately I don't have anything with an Atom CPU. It doesn't happen on the only x86 linux machine I have (a laptop with a first generation AMD Ryzen). I don't expect it to happen on ARM but I will try when I have a little time.
The sample rate change warnings are normal. They will come every few seconds, so when it runs fast it has already finished when the first one would have appeared.
The sample rate change warnings are normal. They will come every few seconds, so when it runs fast it has already finished when the first one would have appeared.
This is really odd.
The problem I'm seeing is not specific to the Atom CPU. I first saw it there, but it also happens on my desktop, which is a Windows machine with an i5-9600k, running Linux under WSL2. And the same on an Intel NUC with an i5-5250u. All of those are running Ubuntu Server, so to see if it's an incompatibility with Ubuntu I tried setting up an Arch Linux VM and am getting the same behaviour there.
Could let me know what version of Linux are you running? I can try setting that up and see if the problem goes away.
When I get a chance I'll try to get my head around Linux performance profiling and see if I can track the problem down further.
The problem I'm seeing is not specific to the Atom CPU. I first saw it there, but it also happens on my desktop, which is a Windows machine with an i5-9600k, running Linux under WSL2. And the same on an Intel NUC with an i5-5250u. All of those are running Ubuntu Server, so to see if it's an incompatibility with Ubuntu I tried setting up an Arch Linux VM and am getting the same behaviour there.
Could let me know what version of Linux are you running? I can try setting that up and see if the problem goes away.
When I get a chance I'll try to get my head around Linux performance profiling and see if I can track the problem down further.
Are you by any chance using an AMD CPU? If so that could explain our different results.
Manjaro linux didn't make any difference for me.
Running with "perf stat" gave some interesting information: The "good" performance scenario:
And the"bad" performance scenario:
I'm focusing on v1.0.3 because that way I can see the difference in performance with an identical codebase. The results with v2.0.0a3 are similar to the "bad" scenario.
This shows the number of CPU instructions executed is roughly the same, but the instructions per clock drops from 0.70 to 0.16. In other words, CamillaDSP is functioning the same but the CPU is running less efficiently in the second case. The results above are for the Atom N3700. The pattern was the same on other Intel CPUs though with overall performance being much higher.
I am way out of my depth trying to understand why a change in the config file would lead to this change in CPU performance. I wondered if it might be something to do with spectre/meltdown mitigations as some of those were supposed to have a big performance hit, so I tried setting "mitigations=off" in the kernel command line but that didn't make any difference. Some of the mitigations were also in CPU microcode but that's much harder to roll back to test. Anyway that's just a guess.
I'm getting this performance difference only on Linux. I tested under Windows and didn't see any difference.
As I indicated above, compiling with target-cpu=x86-64-v2 avoids this problem - at least on the scenarios I've tested.
The takeaway seems to be that on Linux machines with slightly older Intel CPUs, target-cpu=x86-64-v2 gives improved and more consistent performance. I don't have anything more recent than a 9th gen CPU, so I don't know what the effect is on more recent CPUs. I'd suggest adding this option to the standard compile options - at least on Linux. That would rule out some very old CPUs but should be compatible with anything since 1st gen Core i5/i7 (2008) and AMD Bulldozer (2011).
Manjaro linux didn't make any difference for me.
Running with "perf stat" gave some interesting information: The "good" performance scenario:
And the"bad" performance scenario:
I'm focusing on v1.0.3 because that way I can see the difference in performance with an identical codebase. The results with v2.0.0a3 are similar to the "bad" scenario.
This shows the number of CPU instructions executed is roughly the same, but the instructions per clock drops from 0.70 to 0.16. In other words, CamillaDSP is functioning the same but the CPU is running less efficiently in the second case. The results above are for the Atom N3700. The pattern was the same on other Intel CPUs though with overall performance being much higher.
I am way out of my depth trying to understand why a change in the config file would lead to this change in CPU performance. I wondered if it might be something to do with spectre/meltdown mitigations as some of those were supposed to have a big performance hit, so I tried setting "mitigations=off" in the kernel command line but that didn't make any difference. Some of the mitigations were also in CPU microcode but that's much harder to roll back to test. Anyway that's just a guess.
I'm getting this performance difference only on Linux. I tested under Windows and didn't see any difference.
As I indicated above, compiling with target-cpu=x86-64-v2 avoids this problem - at least on the scenarios I've tested.
The takeaway seems to be that on Linux machines with slightly older Intel CPUs, target-cpu=x86-64-v2 gives improved and more consistent performance. I don't have anything more recent than a 9th gen CPU, so I don't know what the effect is on more recent CPUs. I'd suggest adding this option to the standard compile options - at least on Linux. That would rule out some very old CPUs but should be compatible with anything since 1st gen Core i5/i7 (2008) and AMD Bulldozer (2011).
Yes I am using an AMD CPU. I don't have any Intel machine where I could test this, but if just setting target-cpu=x86-64-v2 solves it then this seems like a good idea.
Is anyone here running CamillaDSP on an older Atom CPU? Switching to x86-64-v2 would exclude Atoms made before 2014, for example the Atom N2700 & N2800, z2xxx etc.
Is anyone here running CamillaDSP on an older Atom CPU? Switching to x86-64-v2 would exclude Atoms made before 2014, for example the Atom N2700 & N2800, z2xxx etc.
@spfenwick : Please do I understand correctly that if the pipe does NOT do the gain multiplication https://github.com/HEnquist/camilla...cc37f598542d3743f030/src/basicfilters.rs#L289 as the first operation, the load increases 10 times?