Beyond the Hype: What Independent Benchmarks Say About SolarWinds Database Observability
Database monitoring tools often make the same big promises: “Gain full visibility,” “reduce MTTR,” or “eliminate alert fatigue.”
If you're a DBA, system engineer, or IT leader, you've probably heard those claims many times. What matters more is what happens when a monitoring agent is installed on a production grade system and pushed under real load.
That's what independent technology analyst firm AIMultiple set out to test. The team ran hands on benchmarks on clean systems using MySQL and MongoDB 7.0. The work focused on setup, resource use, dashboard quality, query visibility, and metric behavior under load. It also included a 26 GB import and high volume insert workloads. The two original source articles are MongoDB Monitoring: SolarWinds vs New Relic vs Datadog and MySQL Monitoring: SolarWinds vs New Relic vs Datadog.
What AIMultiple Tested
AIMultiple installed SolarWinds Database Observability, New Relic, and Datadog on clean systems and documented the setup process step by step.
One benchmark focused on MongoDB 7.0. It measured setup time, usability, resource use, and dashboard usefulness under load. The other benchmark focused on MySQL. It compared setup, dashboard quality, query analysis, metric accuracy, and behavior during a 26 GB import test.
Executive Summary
Across the two tests, SolarWinds stood out for fast setup, accurate operation tracking in the MySQL test, and deeper query level visibility than the standard New Relic and Datadog integrations tested by AIMultiple.
|
Operational Metric |
SolarWinds |
New Relic |
Datadog |
|---|---|---|---|
|
MongoDB setup time |
~5 minutes |
~15 minutes |
20+ minutes |
|
MySQL setup time |
8 minutes |
8 minutes |
12 minutes |
|
MySQL operation count accuracy |
5,000 / 5,000 |
3,847 / 5,000 |
Not reported in UI |
|
Query profiling in tested integration |
Available |
Not available in tested integration |
Not available in tested integration |
|
Memory tracking during 26 GB MySQL import |
Detected near full RAM usage |
Reported about 10 percent RAM |
Detected about 16 GB RAM used |
1. Setup Experience
In the MongoDB benchmark, SolarWinds completed setup in under five minutes. It also detected the previously installed agent automatically. The interface showed the operating system, cloud instance ID, and agent version on the selection screen. It then provided three copy and paste MongoDB commands to create the monitoring user, grant privileges, and set profiling.
By comparison, New Relic took about 15 minutes in the same MongoDB test. AIMultiple noted extra friction from manual checks. These included selecting the operating system, checking the MongoDB version manually, and dealing with dashboard confusion around different MongoDB integration options.
Datadog took more than 20 minutes in the MongoDB test. The reviewer described the process as heavily manual. It included YAML editing, command line restarts, and troubleshooting a formatting error before the dashboard populated correctly.
For MySQL, AIMultiple reported setup times of 8 minutes for SolarWinds, 8 minutes for New Relic, and 12 minutes for Datadog.
2. Metric Accuracy Under Load
The clearest accuracy comparison appears in the MySQL benchmark. During the 26 GB import test, AIMultiple reported that SolarWinds tracked all 5,000 operations correctly. New Relic reported 3,847 out of 5,000. Datadog didn't present a comparable operation count in the UI during that test.
The same MySQL workload also showed a major difference in memory reporting. SolarWinds and Datadog reflected the heavy RAM consumption during the import. New Relic reported only about 10 percent memory use.
Those differences matter. If operation counts or memory pressure are understated, troubleshooting and capacity planning become less reliable.
3. Resource Consumption
The MongoDB benchmark also measured agent resource use while all three tools collected data from the same MongoDB instance under load. Overall, all three stayed below a level that would materially strain the system. Total CPU and memory impact remained under 10 percent combined during the test.
On the 16 GB MongoDB test server, memory use was reported as about 90 MB for New Relic, 330 MB for Datadog, and 500 MB for SolarWinds.
AIMultiple also noted that SolarWinds performed significantly more disk reads than New Relic. It described SolarWinds as performing about 40 times as many reads as New Relic and about 1.5 times as many as Datadog.
AIMultiple interpreted that trade off as part of a more detailed query level analysis approach.
4. Query Visibility and Troubleshooting Depth
Across both tests, SolarWinds is the only platform that AIMultiple describes as providing query level profiling in the default evaluated experience.
In the MongoDB benchmark, AIMultiple highlighted query pattern visibility, filterable query diagnostics, response time data, and health status changes when MongoDB was stopped.
In the MySQL benchmark, AIMultiple again pointed to query profiling and diagnostic depth as a differentiator.
The tested New Relic and Datadog integrations focused more on higher level infrastructure and database metrics. According to AIMultiple, they didn't provide the same query level visibility in the default evaluated setup.
There's one important nuance in the MongoDB article. AIMultiple says its Datadog assessment was based on the basic MongoDB integration tested at that time. The article also notes that Datadog later released Database Monitoring for MongoDB with deeper capabilities. That later update is mentioned in the source article itself, but it wasn't the version under test.
5. Security and Configuration Observations
The MongoDB article also includes a security note about MongoBleed. AIMultiple describes it as a severe unauthenticated out of bounds read vulnerability affecting unpatched MongoDB Server versions prior to specified releases. The article advises organizations running MongoDB to move to patched versions.
The same MongoDB setup walkthrough also notes that SolarWinds and New Relic both provided explicit commands for creating monitoring users with limited permissions. That's different from asking users to operate with unrestricted root style access in the database itself.
Final Verdict
Using only the two original AIMultiple articles, the evidence supports a narrower but still strong conclusion.
SolarWinds performed best in setup speed for MongoDB. It matched the fastest setup time for MySQL. It also delivered the clearest query level visibility in the tested experiences and showed the strongest operation count accuracy in AIMultiple’s MySQL benchmark.
That doesn't mean every capability was stronger in every category. It also doesn't justify claims beyond what AIMultiple directly measured. Still, based strictly on those two articles, SolarWinds comes through as the most database focused option of the three products tested in those specific benchmark scenarios.
To explore the SolarWinds database portfolio further, see https://www.solarwinds.com/database.
