Quite a long post today I'm afraid !
Over the last few weeks we've been kicking the tires on the AppFirst monitoring service, and I thought it might be helpful to others to document my thoughts here. I didn't see many such posts when we set out on the task and it may be useful for someone, at least.
Our experience with AppFirst has been really rather bad and while I'm trying to stay positive I'm afraid this posting is going to be largely a rant as I've rarely dealt with such a poor offering from a company. We really have seen an awful lot of issues with the general quality control and design decisions of AppFirst.
Documentation
Firstly, there's the documentation. I ran into typos and mistakes constantly within the first few minutes of using the product. I can only assume no-one at AppFirst actually bothers to check this stuff. For the standpoint of the architecture and usage of the solution it's very lacking so it's hard to comment much on its quality. However the API documentation is quite poor. We've seen API attributes badly described, attributes listed multiple times, and examples of API usage which just plain don't work. I've raised all of these issues with their support, but to now few have been fixed.
To me, this shows a general lack of quality control on their product. After all, this is meant to be sold to sysadmin types - people who typically have a high attention to detail.
The API
Apart from the API specific documentation being littered with mistakes and typographical errors, the API itself is pretty poorly designed. Some issues are minor, like that you need to pass non URL-encoded json into some requests, with no way to just add those elements later on.
Other issues, like their API django thread crashing when you use it, which really show just how piss poor the testing of this company is. They apparently don't actually monitor their own monitoring solution ?! Many times I've mentioned these issues to their support and they've looked into it only to find there is indeed an issue which they didn't know about. WTF ?!
Security
Security on AppFirst is unbelievably poor. Firstly, the RPMs come from an unsigned rpm repo, which is pretty poor form. This is then compounded by the rpm installing a /etc/yum.repos.d/ file to ensure there's a nice easy route for this unsigned software to enter your systems.
To make matters even worse on Debian systems they merge their repo definition into 'sources.list' rather than use a 'sources.list.d' file, which makes it just that little harder to use puppet or similar to remove the repo file again.
Oh, and did i mention you can run up a bill on someone's account just by downloading a file from a publicly available URL ? Yeah, that's right - There's zero checking during installation, if you send metrics to them from a machine with someone else'es tenant id which is stored in /etc/Appfirst, someone else gets charged. I found this out by accidentally installing someone else'es rpm via a puppet install. Sorry to whoever got charged !
Lastly, and very much not least is the way it actually goes about monitoring processes. You might imagine that it uses the /proc file-system, maybe a few shell commands to pick up statistics. You might further imagine that it has a few small C coded binaries which pick up some other numbers to graph. You would not, i venture, imagine that it injects a library into nearly every process on the system. Nearly in this case excludes ssh among others - i assume because they know ssh will shit a brick if you do this.
Who in their right mind thought that was a good idea ? It utterly breaks SELinux compatibility, but fear not ! AppFirst have a workaround for SELinux, although they do say SELinux is not officially supported. You just have to white-list every binary that AppFirst wants to talk to. Wait, what the fuck did you say ? Did you say "allow this piece of shit virtual-Trojan to hook into every binary on the system" ? Why yes, yes you did. The one from the unsigned repo, which may or may not actually be billing your account or someone else's, and you hope isn't sending anything like passwords back to AppFirst.
Honestly, Fuck. that.
Everything else
There's been so many other little annoyances with this company. Here's some more for you: Their home page crashing, frequently. Links in their UI which went straight to 404 pages. Every user of an account having the same API token, which helpfully you can see via their API (WTF, why ?!). There's also quite a lot of UI bugs and glitches - issues like dropdowns which aren't wide enough to even display what's in them. Oh, and the android app is also littered with bugs, at least one of which makes it crash when you want to view a graph.
Overall
Overall I feel compelled to write this blog post because I think when a company offers a product this bad, it's only fair that people give them a hard time about it.