netdata 1.5 released - big update!
-
netdata provides distributed monitoring.
Traditional monitoring solutions centralize all the data to provide unified dashboards across all servers. Before netdata, this was the standard practice. However it has a few issues:
- due to the resources required, the number of metrics collected is limited.
- for the same reason, the data collection frequency is not that high, at best it will be once every 10 or 15 seconds, at worst every 5 or 10 mins.
- the central monitoring solution needs dedicated resources, thus becoming "another bottleneck" in the whole ecosystem. It also requires maintenance, administration, etc.
- most centralized monitoring solutions are usually only good for presenting statistics of past performance (i.e. cannot be used for real-time performance troubleshooting).
Netdata has a different approach:
- data collection happens per second
- thousands of metrics per server are collected
- data do not leave the server they are collected
- netdata servers do not talk to each other
- your browser connects all the netdata servers
Using netdata, your monitoring infrastructure is embedded on each server, limiting significantly the need of additional resources. netdata is blazingly fast, very resource efficient and utilizes server resources that already exist and are spare (on each server). This allows scaling out the monitoring infrastructure.
However, the netdata approach introduces a few new issues that need to be addressed, one being the list of netdata we have installed, i.e. the URLs our netdata servers are listening.
To solve this, netdata utilizes a central registry. This registry, together with certain browser features, allow netdata to provide unified cross server dashboards. For example, using the latest git version of netdata, when you jump from server to server using the my-netdata menu, several session settings (like the currently viewed charts, the current zoom and pan operations on the charts, etc) are propagated to the new server, so that the new dashboard will come with exactly the same view.
Found Here: https://github.com/firehol/netdata/wiki/mynetdata-menu-item
-
Yes, I've read that. It didn't explain why I'd this. It sounds like netdata is just "giving up" rather than providing a solution. It's just a nice, graphical top command? I've got that with top already. But is this getting me that I don't already have? Centralization is the key goal. THe last thing I want to do is have to expose and secure every machine individually, but a pain and risk. I guess I'm missing the "here is the good part" about it, other that the nice looking display.
I did see that you can tie this INTO a central system, but once you have that central system, what is the purpose of netdata?
-
How do I see my servers taht are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them? The purpose of a central console is so that I have one place, one secured place, to go view them. If each machine has its own dashboard, I have to get every one of them out on the Internet so that I can view them?
-
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers that are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them?
From the central registry.
-
@aaronstuder said in netdata 1.5 released - big update!:
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers that are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them?
From the central registry.
So that acts like a proxy server? What does its interface look like?
-
@scottalanmiller said in netdata 1.5 released - big update!:
@aaronstuder said in netdata 1.5 released - big update!:
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers that are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them?
From the central registry.
So that acts like a proxy server? What does its interface look like?
that's not the way I read it. The central registry appears to just be a list of what other machines you can 'surf' to.
-
@Dashrender thats correct
-
@aaronstuder said in netdata 1.5 released - big update!:
@Dashrender thats correct
But this doesn't answer Scott's question - If I'm at home, how do I access servers I won't want published to the internet?
-
@Dashrender said in netdata 1.5 released - big update!:
@aaronstuder said in netdata 1.5 released - big update!:
@Dashrender thats correct
But this doesn't answer Scott's question - If I'm at home, how do I access servers I won't want published to the internet?
SSH jumpbox?
-
@dafyre said in netdata 1.5 released - big update!:
@Dashrender said in netdata 1.5 released - big update!:
@aaronstuder said in netdata 1.5 released - big update!:
@Dashrender thats correct
But this doesn't answer Scott's question - If I'm at home, how do I access servers I won't want published to the internet?
SSH jumpbox?
sure, but that's outside the scope of the product/project, making the project just that much harder and less worth while.
As scott said, if you're already managing a central stat server, you gain your secure access to it, and it shows you everything for all servers, so what does this project do for you?
tracking those thousands of collection points is probably not needed in most production environments - if it was, then the company would probably have already solved that issue.
-
@aaronstuder said in netdata 1.5 released - big update!:
@Dashrender thats correct
So they would be blocked, right?
-
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers taht are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them? The purpose of a central console is so that I have one place, one secured place, to go view them. If each machine has its own dashboard, I have to get every one of them out on the Internet so that I can view them?
I can't even think of 1 server (that I have) where I would this level and speed of real time data let alone 1,000. I know they are out there but I know I don't have any. SNMP and collectors give me way more info than I can use and allow for central monitoring. This seems like a super niche product that could have security implications.
-
@stacksofplates said in netdata 1.5 released - big update!:
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers taht are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them? The purpose of a central console is so that I have one place, one secured place, to go view them. If each machine has its own dashboard, I have to get every one of them out on the Internet so that I can view them?
I can't even think of 1 server (that I have) where I would this level and speed of real time data let alone 1,000. I know they are out there but I know I don't have any. SNMP and collectors give me way more info than I can use and allow for central monitoring. This seems like a super niche product that could have security implications.
On Wall St. we needed this from time to time, but we wouldn't want it running or exposed normally. Very rare, even there, though. Only .1% of servers, literally. For normal needs, what we need is roughly:
- Centralized viewing
- Security
- Data collection away from the source device
- Historical viewing
-
@aaronstuder said in netdata 1.5 released - big update!:
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers that are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them?
From the central registry.
So can anyone see my data? This is the point I don't get. Why would I want my servers uploading data to a "Dashboard" anyone can see.
I want to monitor my servers ..... just me.
-
@scottalanmiller said in netdata 1.5 released - big update!:
- Historical viewing
I think this is the big one where it fails (excluding security). It's only real time data, and no history. So you need two separate monitoring solutions.
-
@hobbit666 said in netdata 1.5 released - big update!:
@aaronstuder said in netdata 1.5 released - big update!:
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers that are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them?
From the central registry.
So can anyone see my data? This is the point I don't get. Why would I want my servers uploading data to a "Dashboard" anyone can see.
I want to monitor my servers ..... just me.
I think the central registry in this case is something you build, something you control - but I could be wrong.
and if I am wrong - holy cats - you're right and that's crazy!! -
@Dashrender said in netdata 1.5 released - big update!:
@hobbit666 said in netdata 1.5 released - big update!:
@aaronstuder said in netdata 1.5 released - big update!:
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers that are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them?
From the central registry.
So can anyone see my data? This is the point I don't get. Why would I want my servers uploading data to a "Dashboard" anyone can see.
I want to monitor my servers ..... just me.
I think the central registry in this case is something you build, something you control - but I could be wrong.
and if I am wrong - holy cats - you're right and that's crazy!!So yeah I like the look of it but without knowing where the data goes as well is a NO from me.
-
@Dashrender said in netdata 1.5 released - big update!:
@hobbit666 said in netdata 1.5 released - big update!:
@aaronstuder said in netdata 1.5 released - big update!:
@scottalanmiller said in netdata 1.5 released - big update!:
How do I see my servers that are not on the Internet, for example? Let's say I have 1,000 servers, how do I view them?
From the central registry.
So can anyone see my data? This is the point I don't get. Why would I want my servers uploading data to a "Dashboard" anyone can see.
I want to monitor my servers ..... just me.
I think the central registry in this case is something you build, something you control - but I could be wrong.
and if I am wrong - holy cats - you're right and that's crazy!!Yes, appears to be just wide open and requires each node to be wide open. So a massive security nightmare. You can, of course, always layer security on top yourself, but who wants to manage that mess?
-
Security:
Yes netdata listens on whatever interface has network connectivity, but it does not access anything outside your network to transmit data, and does not forward any ports to itself using UPNP or NAT-PNP. The only way this would be open to the world is if you set up a port forward. It is not "open to the world" if you install it on a public facing server unless you have no firewall set up on the machine. That would be your own problem, and not one for Netdata to solve.
"netdata keeps all the data on the server they are collected." from their Wiki since nobody here read it.
Centralization:
There are already a ton of products out there that harvest monitoring data from agents and put it in a database on a central server. If you like that model, then go use it. This is not that, and was never designed to be that. It uses cookies and other stuff in your browser to see what other Netdata servers you have accessed (with that browser). That data (stored in your own browser) is used by the netdata dashboard on any of the servers to build a jumplist so you can jump to those machines from within the WebUI. If you REALLY want a centralized place to dump data from all machines, you will have to use the features from this latest release to ship data to Grafana, or some other visualization stack. If you bothered to read the release you would know about the Backends they support:
"netdata supports data archiving to backend databases:
Graphite
OpenTSDB
Prometheus
and of course all the compatible ones (KairosDB, InfluxDB, Blueflood, etc)" - from the wikiAgain, this is all information that can be gathered by reading the release, and the front page of the Wiki. I have only used Netdata on one machine so I am no expert so don't ask me to explain things in more detail. Shit, this post was probably too long for you so I'll make a TL;DR for those who have the attention span of a doorknob.
TL;DR RTFM and stop crying.
-
lol
