Announcements and Updates – Page 66

Status

Redundant Switch Fault

Post author By Roller Network
Post date October 4, 2009

On Saturday evening (September 3, 2009) we experienced a short general outage caused by our DNS and SQL cache pool servers rebooting. Normally this wouldn’t be a problem, but the PowerDNS Recursor package didn’t start at boot time, and a lack of DNS meant a general lack of anything useful taking place.

It was odd that some equipment rebooted itself while others didn’t, so after a lot of thinking, we decided that one of the APC 30A redundant switches we use was probably faulted because this isn’t the first time we’ve seen this. We pulled the switch from service (causing another reboot – although if we were right we didn’t want it risking the system anymore) and opened the cover. Inside we found relay contacts with pitting and arching:

The other relays we opened up weren’t nearly as bad, but they still exhibited discoloration and pitting on the contacts. The one in the pictures was loaded between 10 and 15 amps and it’s supposed to be rated for 30 (or 24 derated). Because this is the second failure we’ve had with this device we’ve decided to remove the remaining ones from service as they are likely to suffer the same fate in the future.

We apologize for the recent bumps in the normally smooth operation you’ve come to expect from us. We understand that your mail and DNS service is important to you, and to us, since we use the same services for our mail. As such, a discount/credit will be forthcoming.

IPv6

Verizon Refuses to Provide Complete IPv6

Post author By Roller Network
Post date October 3, 2009
18 Comments on Verizon Refuses to Provide Complete IPv6

UPDATE: Verizon Post-Slashdot Followup

The final word on Friday from Verizon is that they refuse to carry 29% of the IPv6 internet that is visible from their competitors. We calculated this percentage by taking the total number of paths Verizon provides and dividing it by the number of visible endpoints through Sprint and Hurricane Electric.

Specifically, they are completely blocking all of ARIN’s 2620:0::/23, so even by following their policies they’re still providing an incomplete view of the internet. It is their position that this is “correct”:

“If you wish your /48 to be visible globally, you’ll need to return your direct /48 allocation to ARIN and obtain a Verizon /48 from our network pool. Since our /48 assignment would be part of a /32 that we are announcing, your network would be globally routable. Otherwise, you are limited to AS701.”

Our response was:

Basically it seems to come down to 701 is a black hole for ARIN’s 2620:0::/23 (/40-/48) assignments. Let’s ignore my initial problem for a minute. Even if I were to switch to a /48 contained in your /32 or get my own /32, the following routes are still not available via 701:

Network          Next Hop
*>i2620:0:30::/48   2620:0:950::242:129
*>i2620:0:70::/48   2620:0:950::242:129
*>i2620:0:80::/48   2620:0:950::242:129
*>i2620:0:90::/48   2620:0:950::242:129
*>i2620:0:B0::/48   2620:0:950::242:129
*>i2620:0:C0::/48   2620:0:950::242:129
*>i2620:0:110::/48 2620:0:950::242:129
*>i2620:0:140::/48 2620:0:950::242:129
*>i2620:0:150::/48 2620:0:950::242:129
*>i2620:0:230::/48 2620:0:950::242:129
*>i2620:0:240::/48 2620:0:950::242:129
*>i2620:0:280::/48 2620:0:950::242:129
*>i2620:0:2D0::/48 2620:0:950::242:129
*>i2620:0:310::/48 2620:0:950::242:129
*>i2620:0:320::/48 2620:0:950::242:129
*>i2620:0:330::/48 2620:0:950::242:129
*>i2620:0:380::/48 2620:0:950::242:129
*>i2620:0:380:2::/64
2620:0:950::242:128
*>i2620:0:3F0::/48 2620:0:950::242:129
*>i2620:0:630::/48 2620:0:950::242:129
*>i2620:0:6B0::/48 2620:0:950::242:129
*>i2620:0:6C0::/48 2620:0:950::242:129
*>i2620:0:6C1::/48 2620:0:950::242:129
*>i2620:0:6C2::/48 2620:0:950::242:129
*>i2620:0:800::/47 2620:0:950::242:129
*>i2620:0:802::/48 2620:0:950::242:129
*>i2620:0:830::/48 2620:0:950::242:129
*>i2620:0:860::/46 2620:0:950::242:129
*>i2620:0:862::/48 2620:0:950::242:129
*>i2620:0:870::/48 2620:0:950::242:129
*>i2620:0:880::/48 2620:0:950::242:129
*>i2620:0:8F0::/48 2620:0:950::242:129
*>i2620:0:930::/48 2620:0:950::242:129
*>i2620:0:950::/48 2620:0:950::242:130
*>i2620:0:960::/48 2620:0:950::242:129
*>i2620:0:9B0::/48 2620:0:950::242:129
*>i2620:0:9C0::/48 2620:0:950::242:129
*>i2620:0:A00::/48 2620:0:950::242:129
*>i2620:0:A00::/43 2620:0:950::242:129
*>i2620:0:A01::/48 2620:0:950::242:129
*>i2620:0:A02::/48 2620:0:950::242:129
*>i2620:0:A03::/48 2620:0:950::242:129
*>i2620:0:A04::/48 2620:0:950::242:129
*>i2620:0:A05::/48 2620:0:950::242:129
*>i2620:0:A06::/48 2620:0:950::242:129
*>i2620:0:A07::/48 2620:0:950::242:129
*>i2620:0:A09::/48 2620:0:950::242:129
*>i2620:0:A0D::/48 2620:0:950::242:129
*>i2620:0:A10::/48 2620:0:950::242:129
*>i2620:0:A16::/48 2620:0:950::242:129
*>i2620:0:A17::/48 2620:0:950::242:129
*>i2620:0:A1A::/48 2620:0:950::242:129
*>i2620:0:A1C::/48 2620:0:950::242:129
*>i2620:0:A1D::/48 2620:0:950::242:129
*>i2620:0:A1E::/48 2620:0:950::242:129
*>i2620:0:A1F::/48 2620:0:950::242:129
*>i2620:0:B10::/46 2620:0:950::242:129
*>i2620:0:B60::/48 2620:0:950::242:129
*>i2620:0:C30::/48 2620:0:950::242:129
*>i2620:0:C80::/48 2620:0:950::242:129
*>i2620:0:CA0::/48 2620:0:950::242:129
*>i2620:0:CC0::/47 2620:0:950::242:129
*>i2620:0:CC0::/44 2620:0:950::242:129
*>i2620:0:CCA::/48 2620:0:950::242:129
*>i2620:0:CCB::/48 2620:0:950::242:129
*>i2620:0:CCC::/48 2620:0:950::242:129
*>i2620:0:CCD::/48 2620:0:950::242:129
*>i2620:0:CCF::/48 2620:0:950::242:129
*>i2620:0:CF0::/48 2620:0:950::242:129
*>i2620:0:D20::/48 2620:0:950::242:129
*>i2620:0:D60::/46 2620:0:950::242:129
*>i2620:0:DA0::/48 2620:0:950::242:129
*>i2620:0:DC0::/48 2620:0:950::242:129
*>i2620:0:DD0::/48 2620:0:950::242:129
*>i2620:0:DF0::/48 2620:0:950::242:129
*>i2620:0:E50::/48 2620:0:950::242:129
*>i2620:0:EF0::/48 2620:0:950::242:129
*>i2620:0:1000::/48 2620:0:950::242:129
*>i2620:0:1002::/48 2620:0:950::242:129
*>i2620:0:101E::/48 2620:0:950::242:129
*>i2620:0:1040::/48 2620:0:950::242:129
*>i2620:0:105D::/48 2620:0:950::242:129
*>i2620:0:105E::/48 2620:0:950::242:129
*>i2620:0:105F::/48 2620:0:950::242:129
*>i2620:0:1080::/48 2620:0:950::242:129
*>i2620:0:10A0::/48 2620:0:950::242:129
*>i2620:0:10A1::/48 2620:0:950::242:129
*>i2620:0:1700::/45 2620:0:950::242:129
*>i2620:0:1A10::/48 2620:0:950::242:129
*>i2620:0:1A50::/48 2620:0:950::242:129
*>i2620:0:1B00::/48 2620:0:950::242:129
*>i2620:0:2220::/48 2620:0:950::242:129
*>i2620:0:22F0::/48 2620:0:950::242:129
*>i2620:0:2830::/48 2620:0:950::242:129
*>i2620:0:2860::/48 2620:0:950::242:129
*>i2620:0:2890::/48 2620:0:950::242:129
*>i2620:0:2B10::/48 2620:0:950::242:129

That’s not exactly something to brush off (ignoring the random /64); that’s a big enough chunk missing to make 701 look undesirable to someone like me who wants to heavily promote IPv6. Compared to my existing IPv6 table, 701 is missing 29% of the IPv6 internet that I can already reach.

And based on their position, they’re probably (although we have not confirmed, but based on the 29% figure we came up with it is extremely likely) missing similar ranges from the other regional registries.

IPv6

Verizon Update

As mentioned before, our Verizon circuit wasn’t provisioned properly. After three conference calls and several weeks of calls, moving the endpoint city and re-engineering for latency issues, they’re finally ready to give it another shot today. We will post an update after today’s turn-up conference call.

[14:07] The endpoint was moved back to California (San Jose this time instead of Sacramento) because Phoenix was too slow.

[14:32] IPv4 is up. BGP for IPv4 is up.

[14:53] IPv6 is up without tunnels!

[15:33] Our IPv6 BGP route announcements aren’t working through Verizon – we’re stopped for today and the installation engineer is going to confer with a different group tomorrow.

Announcements

Updated Billing Policy

Post author By Roller Network
Post date September 30, 2009

We recently had a free account holder send us a nasty email accusing us of being scam artists with regards to our billing policy and demanded their account be closed. As such, we have updated our billing policy to address their misconceptions. In our entire history we have never actually applied a late fee to an account and we’re always happy to work with our customers to address their concerns.

The billing policy late fees section now states that anything provided under Personal, Personal Plus, Standard, Standard Plus, Professional, and Professional Plus are exempt from late fees. In addition, we added a cap to the late fees to prevent them from exceeding the invoiced amount and exempted invoices under $100 from the fixed fee. (But, as mentioned before, we haven’t actually ever used the late fees before, so this doesn’t really affect anyone.)

http://www.rollernet.us/policy/billing.php

Uncategorized

New Facility Progress #9

Post author By Roller Network
Post date September 28, 2009

Today the distribution panels were mounted and the conduit attached to the UPS.

All of the conduit and panel installations are just about done, at which point we’ll be waiting for a custom 400A panelboard to arrive in order to finish it out and power everything up.