Cisco FTD device with high volume of event data can prevent policy deployment – (solution found)

8 Comments

A large customer with 10Gig interfaces on their Cisco 4100 FTD’s and 4500 FMC found an issue when bringing new FTD devices online. The issue arose  because there was so much event data the FMC couldn’t push policy to the FTD devices – even with 10Gig interfaces!

  • One solution for this is to increase the timeout on all FTD devices using the FTD CLI:

    1) SSH into the FTD management IP
    2) Enter ‘expert’, followed by ‘sudo su -‘ to elevate privileges.
    3) Enter the following two commands to increase timeouts:

echo ‘file.stream.timeout=1200’ > /ngfw/etc/sf/ccm.properties

echo ‘download.timeout=1300’ >> /ngfw/etc/sf/ccm.properties

4. restart ngfwManager using the command below:

pmtool restartById ngfwManager

 

  • A better solution would be to create separate event interfaces on all FTD devices and the FMC(s), however, understand that this is a very expensive solution:

First, you can see the management interfaces on your FTD devices that you created with Chassis Manager, and are now available with the show network command:

[output cut]

==================[ management0 ]===================
State                     : Enabled
Channels                  : Management & Events
[output cut]

==================[ management1 ]===================
State : Disabled
Channels : Management & Events

[output cut]

Key takeaway: The management interface configured via the FCM is always management0.  The interface designated as firepower-eventing will always be management1 regardless of which SFP slot it is installed into.

After you assign the event interface to the logical device, this interface is not enabled or configured with network settings, and you must go to each FTD CLI separately to configure the interface.

Here is how you configure the management interface on the FTD device:

>configure network ipv4 manual <mgmt 1 IP> <netmask> <gateway> management1

Management1 is enabled automatically with the above command,  but then you need to disable the management channel so you ‘re only sending and receiving events on this new interface.

> configure network management-interface disable-management-channel management1

Just make sure you remove the management channel from the event interface AFTER configuring the IP address.  It won’t work the other way around.

Now, you’ll see this:

[output cut]

==================[ management0 ]===================
State                     : Enabled
Channels                  : Management & Events
[output cut]

==================[ management1 ]===================

State                     : Enabled
Channels                  :  Events
 [output cut]

 

Key Fix not in Cisco’s documentation:  There is still one more step to make this work! Not found in any documentation!

Once the event interface on the FTD has been configured, a static route is required on the FMC such that the FMC will use the event interface’s physical connection for any traffic to the IP of the FTD event interface. This can be configured from within the FMC GUI System > Configuration > Management interface

 

You need to have a lot of data to see this problem pop up! This particular customer has over 30,000 users on their network.

Merry Christmas!

 

8 Comments

  1. I wish this had worked because I have FTD’s in China that i can’t deploy to but ccm.properties does not exist in that directory so not sure how the deploy process would use those parameters.

  2. Hi Todd..

    Talking about bandwidth. Is there any information on how much traffic or bandwidth does a Managed Device (ASA with Firepower module) will send to FMC to log all the security events? I know this will depend on the network size, number of users, etc… but is there any formula to calculate that bandwidth consumption? The thing is that the managed device (sensor) is located in a branch office connected via MPLS link. There is a question similar to this one here: https://community.cisco.com/t5/firepower/firepower-sensor-distribute-deployment/td-p/3298522

    Somebody said:
    Bandwidth up from sensor to the managing FMC can vary greatly. Event reporting will consume, on average, 700 bytes/event. So that’s 5600 bits x your anticipated number of events per second (EPS).

    thanks!

    1. it will greatly on your configuration for logging in the ACP rules….the highest FMC can only take 20,000 EPS…so you need to think about that when transferring packets from the device (it will transfer all snort event packets by default, which is good), as well as logging on the ACP configuration.

  3. Hi Todd,
    I have one customer want with two FMC 2600 in HA and 2 FTD 4100 and want to use separate event and management interface.

    I have some questions please
    1. Does the event and Management interface need to be in the same subnet or different subnet on FMC?

    2. How The FTD will know the event interface IP of the FMC so it can send the event to it ? Because FTD know the management IP of the FMC during the registration but how it will know the event interface?

    3.does the active and standby FMC will share the same ip of the event interface or each one will have separate IP ?

    4.

    1. With a hardware FMC you can separate mangement VLANs and have more than one, but also you can separate management and event traffic. If you go into Settings>Management you can open the interfaces and dictate which interface will receive management traffic and which will receive events.
      Yes, keep everything in the same management subnet. You cannot configure the primary and secondary separately, you can only tell the primary to update the secondary and you cannot make any changes on the secondary.

  4. Thanks, for your reply.
    I will add one more query please.

    2. you mean the physical management interface and physical event interface on FMC and FTD all will be in the same subnet.? If yes, then I just need one default route one the FMC?

    3. For FMC in HA, when I use separate management and event interface. I know that management IP for primary and secondary FMC will be different IP. so, if the primary FMC Managment IP will be (10.0.0.1) and secondary FMC management Ip will be (10.0.0.2).
    then what about the event-interface for primary FMC and secondary FMC? both will share same IP or different IP.

    4.For FTD, how I will tell the FTD to send the event traffic to the event-interface of the FMC? How the FTD know the event interface on FMC?

    1.Does the backup and restore will work on the FMC for different models? I have FMC 2600 and I will migrate to FMC 4600?

    5. does the Evenet-interface on the FTD will be used only to send event logs to FMC, or it will be used for syslog and SNMP also it these will use the management not the Event-interface

    1. Hi Todd,

      now I have two HA FMC 4700 and FTD 4245 as managed devices and i a am using one interface on FMC for management and event and one interface on FTD for management and event. I want to add separate interface on both FTD and FMC.
      could you Please, tell me how i cam proceed with this. and this change will require down time? and should i start with FMC or FTD?
      . one more point if i made the management and event-interface in same subnet how the FTD will detect the FMC event interface?

Comments are closed.