BCC log levels and meaning

All BCC services have the ability to write system events at 4 defined levels of logging. Log settings are configured within the IMC and can be found under "Configuration --> Service Configuration --> Logging Information". In addition to setting log levels, the user can also define the maximum log file size, telnet logging (the ability to view real-time updating of the log, which is generally not useful in daily troubleshooting), as well as flags that will alter the behaviour of the logging service. The flags that may be defined are as follows:

• Always flush logs: Log information is not buffered in memory, rather it is immediately written to disk. This will result in slower performance but will ensure that logs are not lost if the system crashes.
• Archive logs: Creates an archive of saved log files based on the maximum logfile size.
• Write packet info (Vestigial): When this parameter is checked the service log the packet data sent and received on the network. This is generally not useful to our troubleshooting as the same data can be obtained by performing a packet trace. This was used more in Windows installations no longer supported

Normally the service maintains two log files, named service0.log and service1.log, with service1.log being the most recent file. When they are both filled to the maximum logfile size, the older log (service0.log) is destroyed, service1.log is renamed to service0.log and a new logfile named service1.log is created. The exception to this is when "Archive logs" is enabled. Written log files exist in the service 'data' folder:

• Solaris/Redhat: /usr/local/lib/service/data
• Debian: /usr/lib/service/data

Log Levels
Off: Nothing is logged at all except ERROR WARN
0: Minimal logging that only records essential messages (i.e., service start/stop, DHCP records 1 line for each successful DIS/OFF, REQ/ACK, REL)
1: Contains errors in addition to what is contained in level 0. This level is unused mostly in CFM, CFMProxy and MPS.
2: Basic information about queries in addition to what is contained in level 1. CFMProxy logs download success or fail messages, DNS logs successful lookups. This level is unused mostly in CFM and MPS.
3: Verbose information about everything the service does, including API calls. This is the highest level of logging possible and what is primarily used for troubleshooting purposes.

Log Example
Now that you know how to configure the logs, where they are located, and what the different levels are, it's time to look at a file in its written state. Here is an example of a level 3 log of a production system. Notice the very first line of this log is the service name (DHCP), and the version of the service . The log begins with a service already running,

Reading Log Files
The following is the basic format of a log line:
2010.06.09 14:10:23 [AUD0 ] <0001> Message

The elements of the log line are as follows:
2010.06.09: Date
14:10:23: Time
[AUD0]: The log level of the particular line. Could be any one of the following:

• ALL: Displayed when the log level is 'Off' and the service is reloaded, config.txt read
• AUD0, AUD1, AUD2, DBG3: Corresponds to log level
• THRED: Threads that run constantly (i.e., lease janitor)
• WARN: Warning messages
• ERROR: Error messages

<0001>: Thread ID
Message: The particular event that triggered the log line to be written.

Log 'Tags'

DHCP
DHCP log tags appear in the 'message' section of the log after date, time, log level and thread ID. An example of a DHCP log tag is:
(163:REQ:EC040000:00159628A514:R00159628A514): Message
163: Transaction number, the number of packets received by the DHCP service
REQ: Message type. In this case, it's a request, but could be DIS, REQ, LQY, REL, INF
EC040000: Transaction ID
00159628A514: MAC Address of device
R00159628A514: Remote ID, could also appear with prefix of C (Client ID) and clients with no Remote ID just have 'C' by itself

CFM Proxy
An example of a CFM Proxy log tag:
TFTP MODULE::(18:TFTP:10.100.1.20:3427)
TFTP MODULE: Proxy module
18: Transaction number
TFTP: Transaction module
10.100.1.20: Client IP
3427: Client data port

CFM
An example of a CFM log tag:
(2:192.168.76.17:^1/3655A999/)(18:TFTP:10.100.0.20:3427)
2: Transaction number
192.168.76.17: CFM Proxy IP
^1/3655A999/: Dynamic file ID
18: CFM Proxy transaction number
TFTP: File delivery module (TFTP/HTTP/etc.)
10.100.0.20: Client IP
3427: Client data port

Common error messages
The following are common errors that can be found in the service logs:
DHCP
2011.02.04 22:12:37 [DBG3 ] <52FF1940> (67919254:DIS:6CC53100:031C000031C5:C): Rule 'GGSN1_MG-2' Non-Local Relay A Non-Local relay error means that the provisioning service is attempting to assign an IP address from a network that has not been made local to the GIADDR. The configurations found within the "Routing Elements" container (or, less frequently "Adjacent Networks" are used) should be verified against the CMTS running configurations.
2011.02.04 22:12:37 [AUD0 ] <52FF1940> (67919254:DIS:6CC53100:031C000031C5:C): No response for 0.0.0.0 to 10.132.253.129 (No IPs available.). The matching rule for the device does not have any IPs available to allocate.
2010.09.14 14:44:08 [AUD1 ] <AA1E7BB0>>(243588:DIS:4DD39823:54D46FA794CE:R54D46FA794CC): No rules matched. The device did not match the criteria for any of the configured rules