netdata 1.5 released - big update!
-
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers taht are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them? The purpose of a central console is so that I have one place, one secured place, to go view them. If each machine has its own dashboard, I have to get every one of them out on the Internet so that I can view them?
I can't even think of 1 server (that I have) where I would this level and speed of real time data let alone 1,000. I know they are out there but I know I don't have any. SNMP and collectors give me way more info than I can use and allow for central monitoring. This seems like a super niche product that could have security implications.
-
@stacksofplates said in netdata 1.5 released - big update!:
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers taht are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them? The purpose of a central console is so that I have one place, one secured place, to go view them. If each machine has its own dashboard, I have to get every one of them out on the Internet so that I can view them?
I can't even think of 1 server (that I have) where I would this level and speed of real time data let alone 1,000. I know they are out there but I know I don't have any. SNMP and collectors give me way more info than I can use and allow for central monitoring. This seems like a super niche product that could have security implications.
On Wall St. we needed this from time to time, but we wouldn't want it running or exposed normally. Very rare, even there, though. Only .1% of servers, literally. For normal needs, what we need is roughly:
- Centralized viewing
- Security
- Data collection away from the source device
- Historical viewing
-
@aaronstuder said in netdata 1.5 released - big update!:
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers that are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them?
From the central registry.
So can anyone see my data? This is the point I don't get. Why would I want my servers uploading data to a "Dashboard" anyone can see.
I want to monitor my servers ..... just me.
-
@scottalanmiller said in netdata 1.5 released - big update!:
- Historical viewing
I think this is the big one where it fails (excluding security). It's only real time data, and no history. So you need two separate monitoring solutions.
-
@hobbit666 said in netdata 1.5 released - big update!:
@aaronstuder said in netdata 1.5 released - big update!:
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers that are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them?
From the central registry.
So can anyone see my data? This is the point I don't get. Why would I want my servers uploading data to a "Dashboard" anyone can see.
I want to monitor my servers ..... just me.
I think the central registry in this case is something you build, something you control - but I could be wrong.
and if I am wrong - holy cats - you're right and that's crazy!! -
@Dashrender said in netdata 1.5 released - big update!:
@hobbit666 said in netdata 1.5 released - big update!:
@aaronstuder said in netdata 1.5 released - big update!:
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers that are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them?
From the central registry.
So can anyone see my data? This is the point I don't get. Why would I want my servers uploading data to a "Dashboard" anyone can see.
I want to monitor my servers ..... just me.
I think the central registry in this case is something you build, something you control - but I could be wrong.
and if I am wrong - holy cats - you're right and that's crazy!!So yeah I like the look of it but without knowing where the data goes as well is a NO from me.
-
@Dashrender said in netdata 1.5 released - big update!:
@hobbit666 said in netdata 1.5 released - big update!:
@aaronstuder said in netdata 1.5 released - big update!:
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers that are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them?
From the central registry.
So can anyone see my data? This is the point I don't get. Why would I want my servers uploading data to a "Dashboard" anyone can see.
I want to monitor my servers ..... just me.
I think the central registry in this case is something you build, something you control - but I could be wrong.
and if I am wrong - holy cats - you're right and that's crazy!!Yes, appears to be just wide open and requires each node to be wide open. So a massive security nightmare. You can, of course, always layer security on top yourself, but who wants to manage that mess?
-
Security:
Yes netdata listens on whatever interface has network connectivity, but it does not access anything outside your network to transmit data, and does not forward any ports to itself using UPNP or NAT-PNP. The only way this would be open to the world is if you set up a port forward. It is not "open to the world" if you install it on a public facing server unless you have no firewall set up on the machine. That would be your own problem, and not one for Netdata to solve.
"netdata keeps all the data on the server they are collected." from their Wiki since nobody here read it.
Centralization:
There are already a ton of products out there that harvest monitoring data from agents and put it in a database on a central server. If you like that model, then go use it. This is not that, and was never designed to be that. It uses cookies and other stuff in your browser to see what other Netdata servers you have accessed (with that browser). That data (stored in your own browser) is used by the netdata dashboard on any of the servers to build a jumplist so you can jump to those machines from within the WebUI. If you REALLY want a centralized place to dump data from all machines, you will have to use the features from this latest release to ship data to Grafana, or some other visualization stack. If you bothered to read the release you would know about the Backends they support:
"netdata supports data archiving to backend databases:
Graphite
OpenTSDB
Prometheus
and of course all the compatible ones (KairosDB, InfluxDB, Blueflood, etc)" - from the wikiAgain, this is all information that can be gathered by reading the release, and the front page of the Wiki. I have only used Netdata on one machine so I am no expert so don't ask me to explain things in more detail. Shit, this post was probably too long for you so I'll make a TL;DR for those who have the attention span of a doorknob.
TL;DR RTFM and stop crying.
-
lol
-
or go here and start reading.
https://www.reddit.com/r/sysadmin/comments/5pvg5n/netdata_the_opensource_realtime_performance/ -
@cgunzelman said in netdata 1.5 released - big update!:
TL;DR RTFM and stop crying.
No one is crying, we are trying to figure out the good use case for it, especially as its security model goes against modern LANless design and explicitly goes against the normal security approach of the OP, so we are all very confused as to why he likes it and why he would use it and what the benefit is. We've definitely read the manual, and that just solidified our confusion. We were aware that you can collect it into something else, but most of those tools already have their own collection. It's from reading the manual that we are left unsure what the benefits are outside of super low latency monitoring environments that you would not find in most environments.
It seems like an itch without a scratch. But as someone who uses it, what is the use case for distributed, non-centralized monitoring and how are you tackling the security implications of having it be wide open and does that mean you are relying on LAN security alone and not attempting to use this in a hosted environment?
-
@cgunzelman said in netdata 1.5 released - big update!:
or go here and start reading.
https://www.reddit.com/r/sysadmin/comments/5pvg5n/netdata_the_opensource_realtime_performance/"You can run all your netdata behind another web server, like nginx, apache, lighttpd, etc. You can configure authentication at this front-end web server. The wiki has configuration pages for all of them.
All your netdata register themselves to the my-netdata menu of the dashboard, so you can jump from server to server easily. Several dashboard settings are also propagated from server to server (like current section, zoom level, view timeframe, etc)."
So how would that work, we have to build secure channels between every host and an aggregation point? Reddit leaves me more thinking that it is lacking rather than more. I think the issue might be that this is a product that assumes single site, 1990s LAN-based security and most of us assume or plan for LANless security and multi-site by default. So we instantly ask questions, for even the smallest companies, that this assumes won't come up. That's what I've gathered from reading the manual and Reddit. Basically, it's an awesome tool for a legacy world. I don't know any companies that work in that model today, they exist, but I don't run into them any more. Everyone has cloud hosts, data centers, multiple sites or something these days.
-
All of my machines have local firewalls built with FireHOL and ansible to distribute the config. I plan on using netdata on both my public facing servers (their own firewalls) and machines in a LAN.
Offtopic: your video insinuates Webroot has very low overhead, it does not with stock settings. I've got it installed on 80 windows servers and 1500 workstations. We are looking to move away from it simply because of all the issues it caused on terminal servers.
-
@cgunzelman said in netdata 1.5 released - big update!:
All of my machines have local firewalls built with FireHOL and ansible to distribute the config. I plan on using netdata on both my public facing servers (their own firewalls) and machines in a LAN.
But the local firewalls will have to be opened on the monitoring port for netdata to work. What's your plan to tie them together, VPN?
-
@scottalanmiller So none of your clients have site-to-site VPNs? Not for printers? Not for legacy applications? What's terrible about opening the port for the webUI to local machines? I could see forwarding the port to the open world to be questionable for security since this is such a new product.
-
@cgunzelman said in netdata 1.5 released - big update!:
@scottalanmiller So none of your clients have site-to-site VPNs? Not for printers? Not for legacy applications? What's terrible about opening the port for the webUI to local machines? I could see forwarding the port to the open world to be questionable for security since this is such a new product.
None use site to site for everything, which is what would be required. Some site to site would not solve the issue, only total site to site. And no, none have that.
Our customers specifically are even less likely to have it as VPNs are an extension of LAN security and we generally, but not always, move away from that. We very rarely implement it and more often than not remove it (slowly, over time as things are replaced.)
The only issue with opening the port on local machines is that you are investing in trusting your LAN. Your network design requires that the LAN be a trusted location, which it might be today, but it's technical debt based on that design. The issue that all of us commenting have, and all of our customers have, is that they don't have fully trusted LANs for their servers. They might have some or none, but none have full. Without full, this can be used on some servers, but rarely the important ones. Unless we build out a complex security infrastructure of our own to support it.
-
@cgunzelman said in netdata 1.5 released - big update!:
I could see forwarding the port to the open world to be questionable for security since this is such a new product.
There is no authentication, so even if the product was old and battle tested, it would still expose your data to the outside. Assuming I trust the product 100% to do what it is supposed to do, that still wouldn't work. And I'm not doubting the creators did an excellent job, I'm sure they did and it looks like an amazing product that just doesn't fit a need that I can imagine having today.
-
It's like a crazy awesome mouse trap, but none of us have mice
-
I've been using it to monitor a new firewall distribution that we have been testing for a few months now. It's great for real-time data collection, seeing everything in one neat package.
If you want authentication, use a mainstream webserver instead of the built-in. It says so right there in the wiki.
You may not have a need for it in the small business world (MSP I assume) but to others with specialized projects or need realtime monitoring on a simple and easy-to-read dashboard, it's perfect.
-
What I pointed out above is, what I believe, that every business I know requires in this kind of monitoring. Some may not need all of this, but these are the points that are important:
- Centralized viewing
- Security
- Data collection away from the source device
- Historical viewing
Outside of Wall St trading applications, where the overhead of this is likely out of the question, I don't know of anyone needing the level of detail here so the selling point seems empty. But it misses all four needs that they do have in the real world. It might be an amazing top replacement, but I already have top implemented and with full security built in. This does allow you to use DevOps tools and avoid logging in for top, so that's nice. But it introduces so many security complications that it's hard to see that that is offset by that one thing.
