Monitoring with VMware vRealize Log Insight
VMware vRealize Log Insight delivers highly scalable log management with actionable dashboards, analytics, and broad third-party extensibility, giving you deep operational visibility and faster troubleshooting.
Log Insight is particularly useful in multi-cloud and hybrid environments that include cloud-native applications because it provides operational visibility and faster troubleshooting across physical, virtual and cloud environments. Log Insight can process a container's standard output as a data stream.
For a quick overview of how Log Insight integrates with VMware Enterprise PKS, check out the following video:
Instant Integration
VMware Enterprise PKS integrates with vRealize® Log Insight to provide visibility into the core layers of the container platform, giving you pinpoint traceability and monitoring through intelligent data tagging. VMware Enterprise PKS aggregates and ships all logs to Log Insight with searchable tags, such as cluster, pod, namespace and container. VMware Enterprise PKSf, which encrypts log data in transit with SSL, uses log limiting and throttling to prevent overflow or loss of data to the Log Insight endpoint.
Centralized Logging for Kubernetes
Centralized logging is an essential part of any enterprise Kubernetes deployment. Configuring and maintaining a real-time, high-performance central repository for log collection can ease the day-to-day operations of tracking what went wrong and its impact.
Effective central logging also helps development teams observe application logs to analyze performance.
Log Insight can also be set up to work with VMware PKS or the open source version of Kubernetes by using Fluentd; for details, see the blog post titled Forwarding Kubernetes Log to vRealize Log Insight via Fluentd.
Taking the Audit Trail to Compliance
Security compliance and auditing often require a company to maintain digital trails of who did what and when. In most cases, a robust logging solution is the most efficient way to satisfy these requirements.
Out of the box, VMware Enterprise PKS creates a powerful logging layer on top of Kubernetes by using a combination of Fluentd and VMware vRealize Log Insight. This blog post describes how this integration works and how you can leverage it to quickly capture aggregated container logs from your Kubernetes pods and view them in the vRealize Log Insight dashboard.
The following diagram illustrates the integration:
Container Log Format and Log File
By default, container engines such as Docker capture the standard output or error and leverage the JSON-file driver on each host to write messages to files. Docker maintains a separate log file for each container and stores it in the /var/log/containers directory of the Docker host. Annotation for each log entry consists of the following:
Logging Requirements and Options for Cloud Native Applications
Log aggregation requirements are much more than message rendering. An effective log aggregator must support the processing of events from thousands of endpoints, the ability to accommodate real-time queries, and a superior analytics engine to provide intelligent metrics to solve complex technical and business problems.
You have the option to implement log aggregation using vRealize Log Insight or a number of popular open source or commercial logging analytics solutions, such as the following:
Elasticsearch
Fluentd
Kibana
Splunk.
Each solution has a set of strengths and weaknesses. VMware Enterprise PKS gives you the flexibility to let you choose a solution that best aligns with your processes, tooling, and environment.
Conclusion
Centralized logging is a mandatory requirement of an enterprise Kubernetes deployment. The ability to view and filter logs in real time across thousands of endpoints is vital to be able to triage and resolve infrastructure and application issues quickly.
Combining Log Insight with VMware Enterprise PKS is one way to implement logging for Kubernetes deployments. To find out more, take a look at this blog post:
Last updated
