Categories
Status

January 31, 2020: Degraded Service for Wireless Customers

At approximately 14:00 US/Pacific time and lasting for approximately 45 minutes, some wireless customers were affected by a Cisco IOS bug that was causing some prefixes not to be inserted into TCAM on the preferred (shortest path) core router connected to the serving POP, resulting in null routing for affected prefixes.

Once we were aware of the issue the affected core router was removed from service for analysis. Thanks to the fully redundant nature of our core and backhaul design, this allowed all traffic to self-reroute over alternate paths.

The scope of this issue was limited to wireless customers connecting to a specific POP in Sparks, NV. No other services were affected.

Categories
Status

Annual UPS Preventative Maintenance Visit (2019)

Our annual UPS preventative maintenance visit from Eaton for all facility UPS systems will be on Saturday, October 26th beginning at 09:00 Pacific time and lasting until it’s done (usually mid-afternoon).

As with all routine preventative maintenance visits, we do not anticipate any impact to the facility. If necessary, critical loads will be transferred to generator.

UPDATE: Annual maintenance completed on all units with no issues. (Oct. 26 @ 13:53 PDT)

Categories
Changes Status

Rejecting BGP RPKI “Invalid” Prefixes

Roller Network AS11170 will be updating our routing policy to reject any IPv4 or IPv6 prefix with a BGP RPKI validation result of “invalid” on both the peering and transit borders of our network. We’ve been running RPKI validation internally for a while with the “bgp bestpath prefix-validate allow-invalid” setting configured. This routing policy change will simply remove this line from our BGP address family configurations.

Categories
Status

Account Control Center Maintenance

The Account Control Center will be unavailable for about 7 minutes today (May 26, 2019) between 10:50 and 11:00 Pacific time for network maintenance.

UPDATE: Completed at 11:00:16.

Categories
Announcements Changes

Mail Services: Whitelist Behavior Change

We’re changing the behavior of the whitelist for the “All Filters” entry type to now include the Antivirus filter when whitelisting.

The original behavior for the last decade or so has been to continue applying the antivirus filter while whitelisting everything else unless a second whitelist entry was added explicitly for the antivirus filter. Lately we’ve spent too much time explaining this, so we’ve decided the time has come to change the behavior so that an “All Filters” whitelist entry now truly means all filters (including antivirus).