[cf-dev] Logging Infrastructure for CF components

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[cf-dev] Logging Infrastructure for CF components

ronak banka
Hi everyone,

I have some queries regarding persistent storage of application logs and cf component logs .

As per my understanding 

-->For application logs:
we can send the application logs to doppler with help of metron agent and further stream using traffic controller (User Side).

-->For CF component syslog:
We can send cf component syslog via metron to custom syslog endpoint (followed by parsing and other mining stuff)

On the operator side how can we store "Application logs for all the applications" to a persistent storage??

If i look at firehose (or using noaa to get all the logs), it is more like pulling model from end component side so if components before it are slow in sending logs then there will be buffers . 

Application logs are distributed on different doppler nodes based on AZ of metron and doppler itself , so is there a need to implement something in between doppler and traffic controller to push all the application logs ??

Thanks
Ronak Banka

_______________________________________________
cf-dev mailing list
[hidden email]
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
Reply | Threaded
Open this post in threaded view
|

Re: [cf-dev] Logging Infrastructure for CF components

Erik Jasiak
Hi Ronak,

    We're trying to make sure we understand the question here -

* Component logs do not go through loggregator today, only to the rsyslog endpoint designated.

* All app logs would be available to an operator via the firehose.

* " it is more like pulling model from end component side so if components before it are slow in sending logs then there will be buffers" - component logs again are not part of the firehose - even then, there would be buffers, but all messages are timestamped (if you're worried about ordering?)

* "is there a need to implement something in between doppler and traffic controller to push all the application logs "  - not sure I understand, but though you need at least dopper in every AZ that a metron is in today, doppler can speak to traffic controllers cross-AZ.

Did that help?
Erik

On Wed, May 6, 2015 at 7:48 PM, ronak banka <[hidden email]> wrote:
Hi everyone,

I have some queries regarding persistent storage of application logs and cf component logs .

As per my understanding 

-->For application logs:
we can send the application logs to doppler with help of metron agent and further stream using traffic controller (User Side).

-->For CF component syslog:
We can send cf component syslog via metron to custom syslog endpoint (followed by parsing and other mining stuff)

On the operator side how can we store "Application logs for all the applications" to a persistent storage??

If i look at firehose (or using noaa to get all the logs), it is more like pulling model from end component side so if components before it are slow in sending logs then there will be buffers . 

Application logs are distributed on different doppler nodes based on AZ of metron and doppler itself , so is there a need to implement something in between doppler and traffic controller to push all the application logs ??

Thanks
Ronak Banka

_______________________________________________
cf-dev mailing list
[hidden email]
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
[hidden email]
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev