Just a guy in Vermont trying to connect all the dots...
Dan York
Author's posts
Sep 08
Watch UKNOF Today To Learn About IPv6, IXPs, Internet Connectivity
Want to learn about IPv6, Internet Exchange Points (IXPs) and some of the latest connectivity ideas in the United Kingdom? If so, you can watch the live webcast of UKNOF 29 starting today, September 8, 2014, at 13:30 British Summer Time (BST, which is UTC+1). The critical links to know are:
Today’s sessions are mainly focused on network operations and Internet connectivity and include:
- HEAnet’s Optical Backbone & School’s Connectivity
- Watery Wireless
- Options for Metro 100Gig
- Network Function Virtualisation, bringing virtualised network infrastructure into the cloud
- Broadcast editing and delivery over IP (from the BBC)
- LINX’s UK regional peering strategy
- An overview of BT’s network infrastructure in Ireland and Northern Ireland including connectivity to the the rest of the UK
- UKNOF Status Update
Tomorrow, Tuesday, September 9, 2014, the morning sessions include some that may be of interest to our readers and then the afternoon will be our ION Belfast event. Here’s the current UKNOF agenda going from 09:30-12:35 BST:
- Latest Internet Plague: Random Subdomain Attacks (about DNS security)
- Tales of the unexpected – handling unusual DNS client behaviour
- Using 100 Billion DNS Queries to Analyse the Name Collision Problem
- What went wrong with IPv6?
- IPv6-only Data Centres
- Introduction of UK IPv6 Council
In particular I would point your attention to the “What went wrong with IPv6?” talk at 11:30 BST by Dave Wilson from HEAnet. He recently gave a version of this talk at RIPE 68 and both the video and the slides from that talk are available. He asks some great questions and, I think, has some great ideas for we can advance IPv6 deployment – definitely worth listening to!
After a lunch break, our ION Belfast event will then begin with a packed agenda talking about IPv6, DNSSEC and securing BGP. That, too, will be webcast live for all to see!
All in all it will be two days of outstanding sessions talking about the Internet’s infrastructure and how we can make it work better, faster and more secure!
I hope you will join me in tuning in to watch!
Sep 05
New RFC 7344 – Aiming To Solve The DS Upload Issue in DNSSEC
How can we automate the communication between a DNS operator and a registrar when a DNSSEC key has changed?
I saw this issue very starkly myself the other week when I received 4 email messages from one of the DNS hosting operators I use telling me that new Key Signing Keys (KSKs) had been generated for four of my domains. Now, on one level this was good, in that they were automagically rolling the KSKs over without me having to do anything. However… they had no way to tell the registrars for those domains that a new DNSKEY record (and therefore a DS record) had been created. In other words:
The global chain of trust was now broken for those four domains.
Any DNS resolver performing DNSSEC validation would now find a broken chain of trust and would send back a SERVFAIL. People behind a DNSSEC-validating DNS server would not be able to reach my site.
Now, this happens for me because I use a different operator for my DNS servers than my registrar. If you think of the different players involved in the DNSSEC process, very often a registrar is also acting as a DNS hosting operator. In other words, when you register your domain with a registrar, they are also providing the DNS services for you. In that case the DNS hosting side of the company can communicate to the registrar side of the company that there is a new key and all can work well.
However, in my case the company providing DNS services for me is different from my registrar. I am paying a company for DNS hosting – but I could instead be hosting my own authoritative DNS servers. This might be common if I were an enterprise / business that operated my own data centers, for instance.
The key point here is that a registrar needs to pass a Delegation Signer (DS) record (or in some cases a DNSKEY record) up to the registry for the top-level domain (ex., .org, .com, .whatever). This needs to happen in order to have the “global chain of trust” work and to be sure that the DNSSEC signatures are not being falsified.
Within the DNSOP working group in the IETF we’ve been debating a number of proposals about how to fix this for quite some time. One of those proposals has now been issued as an Informational RFC 7344:
http://tools.ietf.org/html/rfc7344
Formally titled “Automating DNSSEC Delegation Trust Maintenance“, the RFC specifies the creation of two new DNS record types that you would use to signal to a parent zone (for example, a top-level domain registry) that you have a new DNSSEC record that the parent zone needs to retrieve. The two records are:
- CDS
- CDNSKEY
Typically you would probably use one or the other depending upon what your TLD registry requires, but both are specified within the RFC 7344.
The RFC goes into much greater detail, but in a nutshell it would work something like this:
- As the DNS operator of EXAMPLE.ORG, I would generate a new DNSKEY record (for the KSK).
- I would also generate a DS record and publish that as a CDS record.
- The parent zone, .ORG in this example, would notice that a new CDS record has been published.
- The .ORG registry would retrieve my CDS record and then publish it as a DS record for my zone.
- Once the DS record has been published I could then stop publishing the CDS record until the next time I made a change.
The global chain of trust would now be intact.
The key challenge of this approach is step #3 – how does the parent zone notice that a new CDS record has been published?
This is a critical point and one that was debated at some length. The primary thinking is that parent zones that want to use this type of approach would create some kind of “parental agent” that would poll zones periodically to see if there are new CDS records out there. RFC 7344 gets into this in section 6.1 and suggests that there could be both polling and pushing mechanisms developed. Such software is not yet out there, but now that the RFC is out it can certainly be developed.
In any event, this RFC 7344 is now out there providing one potential solution to this “DS upload” problem. What do you think? Is this something you can see implementing? Would you like to see your registries, registrars and DNS hosting operators supporting this approach? Do you have another idea?
P.S. Want to get started with DNSSEC? Please visit our “Start Here” page to find appropriate resources…
Sep 05
MarketingPodcasts.com Coming Soon From Jay Baer
Jay Baer is the force behind the site and said last month on Google+ only this:
Soon, I am launching MarketingPodcasts.com the search engine for podcasts about all things marketing and communications.
One of our key features will be podcast reviews (like Pitchfork, for you indie music geeks).
As readers probably know, I am a weekly contributor to the For Immediate Release (FIR) podcast that focuses on the intersection of social media and public relations, business and marketing. I am a huge fan of audio podcasting, and FIR is just one of the podcasts to which I contribute. I also enjoy listening to podcasts... and so I'll be intrigued to see what Jay surfaces through this new site.
Right now you can just provide your email address to be notified when the site goes live. We'll see, hopefully soon, what it is all about!
If you found this post interesting or useful, please consider either:
- following me on Twitter;
- adding me to a circle on Google+;
- following me on App.net;
- subscribing to my email newsletter; or
- subscribing to the RSS feed.
Sep 04
Walk-Through: Skype 5.4 For iPhone Lets You Host Audio Conference Calls
A couple of caveats and thoughts from the testing:
- As Gary Wong noted in his blog post this first release is limited to four people in a group audio call.
- It seems from the testing described below that the "group audio call" is limited to Skype users. I couldn't find a way to add in a call to an external phone number.
- It also seems to be limited to the iPhone and is not yet available on the iPad.
- Image sharing isn't seamless between Skype for iPhone users and Skype desktop users.
- The user interface was a bit troubling when switching between parts of the group call.
With that in mind, here are more details...
An Architecture Change For Audio Conferencing
What's interesting is that this capability is a change from the way that Skype has historically "hosted" audio conferences. With the existing Skype desktop clients, when you launch an audio conference call, your computer does all the mixing of the audio streams.
For this reason, if you want the best quality audio conference (or "group audio call"... I note that Skype is pointedly avoiding using the term "conference call") your smartest plan is usually to find the person with the fastest computer and fastest Internet connection. The combination of those two factors can make your audio call work the best.
Perhaps obviously, as powerful as they are, today's smartphones aren't going to have the CPU or bandwidth to do all the mixing of the audio streams and sending them back out to all participants.
So this new "group audio call" feature from Skype has to be using some audio mixing happening back in servers in Microsoft/Skype's "cloud" (also known as their "central data centers"). Your iPhone then becomes the control center for the group audio call and also sends your audio stream and receives back the mixed audio stream.
Walking Through Group Audio Calls
Naturally I had to try this out and enlisted the help of two long-time fellow testers - Jim Courtney and Phil Wolff.
Part of this new feature is that there is now a "phone" icon at the bottom of every chat window on your iPhone. A simple tap of that phone icon will initiate a group call with "everyone" in that chat. I didn't have Jim and Phil in a group chat smaller than the 4-person limit, so I started out with a regular voice call via Skype to Jim. I then tapped on the "add a person" icon in the lower right and added in Phil. As the call was connecting to Phil, here is what it looked like:
After Phil accepted the call, I could tap on the "multiple person" (or "group") icon at the top and see a list of who was on the call:
Tapping the "star" icon on the bottom would add this to my "Favorites" in the iPhone Skype client. Tapping the "..." button brought up a small set of options:
Choosing to "rename group" let me give it a new name ("Testing Skype") which then appeared at the top of the window:
Now, Jim and I were both using Skype on our iPhones while Phil was connected using Skype on his Mac. Neither Jim nor I could easily figure out how to start a text chat, but as part of the call Phil had a chat open up in his Mac Skype client. Once he typed in that, Jim and I both had a chat window on our iPhone:
Jim and I could then enter in messages in our iPhone clients without any problems. I also had a Mac Skype client open and Jim had a Windows Skype client open and we could see the chat messages there, too, and could type messages in those clients - it all worked fine.
One interesting issue was the support of sharing files or photos across the clients. When Phil dropped a photo into the chat on his Mac desktop client, I was unable to see it in my iPhone:
Tapping the "i" icon next to the message brought me to a page explaining that the iPhone client only used Skype's new "cloud-based" photo sharing service. Curious to explore this more, I tapped the camera icon and shared out an image I had on my iPhone. The result was visible to both Jim and I (and no, that's not me but rather a contractor working on our house):
However, now Phil was not able to see the photo in his Mac desktop client (nor was I) but was instead directed to go to a URL in his browser to see the image:
Phil said on our call that in order to view that photo he had to login to his Skype account. After our 10 minutes or so of testing this, Phil dropped off and navigating back to the "call" screen I could see that he was no longer on the call:
At this point I could have tapped on the green phone icon to bring Phil back in, but we were done.
When this was over, I did now have my new "Testing Skype" chat with Jim and Phil in it - and at the bottom was a phone icon. Jim tapped the phone icon on his iPhone and reconnected all three of us into a call.
Final Thoughts
If I used Skype on my iPhone a great amount, I could see how this feature would be quite useful for initiating group audio calls. I could create a "group" (effectively a "chat") with a group of people and add that to my "favorites". Then I could simply go into my "favorites" on my iPhone client and initiate the call. Obviously the initial four-person restriction limits the usefulness to only small teams/groups right now, but presumably Microsoft/Skype will raise that limit over time as this feature rolls out more.
The audio quality was fine. I didn't see a way to find out the technical details, but the audio sounded high quality, i.e. it was using Silk or another wideband codec to give rich audio.
I did find the navigation to be a bit cumbersome and not intuitive. Switching between the chat window, the "call status" window (showing the participants) and the regular call window was not as easy as I would have liked. It took some poking and tapping to figure out how to move around.
We did wonder why Skype was rolling out this particular feature right now. Phil wondered if there might be competitive pressures with Apple's announcements coming on Tuesday - for example, will we see group audio calls for Apple's Facetime? We'll have to tune in to see!
It also may purely be Skype seeking to reclaim some of the leadership on features for OTT voice apps given that so many other players have entered the market. Whatever the case... the feature is now out there and available for iPhone users.
If you'd like to try this out yourself, you should be able to download an update from the AppStore.
If you have already tried it, what do you think?
An audio commentary on this topic is also available on SoundCloud:
If you found this post interesting or useful, please consider either:
- following me on Twitter;
- adding me to a circle on Google+;
- following me on App.net
- subscribing to my email newsletter; or
- subscribing to the RSS feed
Sep 04
TDYR 172 Testing Skype’s New Audio Conferencing On The iPhone
TDYR 172 Testing Skype's New Audio Conferencing On The iPhone by Dan York
Sep 04
WordPress 4.0 Provides A MUCH Better Editing Experience!
The other features in WordPress 4.0 are also cool. Being able to more easily work with the media library will be nice. Having the embeds automagically appear in the post without needing to preview will also help save time and let you know how the post will look. Improving the plugin directory is nice, too, although right now I'm pretty set with the plugins I need on my various sites.
It's the improved editing experience that I'm really looking forward to using more. I've already upgraded several of my sites and I like the experience so far. Tomorrow I'll upgrade Deploy360 which is where I expect to reap the biggest benefit.
What about you? Have you upgraded yet? Do you like it? (Keeping in mind that there is nothing special about WordPress "4.0" other than that it is the release between "3.9" and "4.1"... i.e. it's not a "big" release but rather just another "regular" WordPress release.)
Here's the WordPress 4.0 release video showing some of the new features:
If you found this post interesting or useful, please consider either:
- following me on Twitter;
- adding me to a circle on Google+;
- following me on App.net;
- subscribing to my email newsletter; or
- subscribing to the RSS feed.
Sep 04
VoiceOps – Mitigating SIP Threats With SBC Policies, Auto-Blacklisting
There’s a good discussion going on right now (September 2014) in the VoiceOps mailing list about how you can mitigate SIP threats by configuring the policies and settings on your session border controller (SBC). It started out with a detailed question from Robert Nystrom asking about how to configure an Acme Packet SBC in the most secure manner and asking about how best to configure access control lists (ACLs). Several answers can be seen in the VoiceOps archive from folks such as Ryan Delgrosso, Mark Lindsey, Jim Gast and Patrick McNeil, offering commentary and suggestions about how best to proceed.
If you are not already subscribed, the VoiceOps mailing list is a great resource. As stated on the subscription page:
This list is for discussions related to managing voice networks, both traditional and IP.
The VOIP Operators’ Group (VOG) charter is to facilitate the creation, maintenance, and operations of Voice over Internet Protocol (VOIP) related networks, products, and services.
Similar to the North American Network Operators’ Group (NANOG), The Voice Operators’ Group seeks to assist in the creation of a robust, stable and growing VOIP ecosystem.
While the topics are definitely not all about security, I would encourage you to join the list if you do anything with the operation of VoIP networks – or if you are just curious to learn more about such networks.
Sep 03
TDYR #171 – The Dark Side Of Being A Prolific Writer
There is a dark side to being a prolific writer or prolific "content creator"... and that is that if you do NOT create content, you start to get a bit twitchy... in this episode I talk about that feeling of having your head exploding with ideas...
Sep 03
The Live Video Streaming Nightmare: What Do You Do When YouTube Won’t Start? (And Lessons Learned)
What do you do if you go to start live video streaming of an event - and the streaming service you use won't start??? How do let people waiting to watch know? How do you fall back to another service?
Last month at the IETF 90 meeting in Toronto I experienced this nightmare of anyone doing live video streaming across the Internet. We had a presentation at noon on Thursday that we had widely publicized in social media and via email. The cameras were all set up. The producer was all ready to do the switching and encoding out to YouTube. Everything was good to go.
Then at 11:45am, I went to do my part in the process. I needed to login to the IETF YouTube account and basically click the "Start Streaming" button inside the "live event". At that point the YouTube servers would accept the encoded stream from the producer's gear and the stream would go live.
BUT... instead I got this message in a bright red bar across the top:
The message read:
YouTube live is undergoing maintenance. Events cannot be created, started or stopped. Events already started will continue to stream uninterrupted.
Yes... we had no way to START the live video stream!
Over the next few minutes I kept refreshing that page but the warning stayed up there. I was getting rather nervous as the launch time approached!
Now, as it happened, we had already planned to have a second live video steam going out for the event through a different service, Livestream.com, for a different reason.
Why The IETF Uses YouTube For Live Video Streaming
To explain a bit more, we have been using Google's YouTube live video streaming for some of the larger IETF plenary sessions for five reasons:1. The live video stream is available over both IPv4 and IPv6.
2. The stream works easily across pretty much all desktop and mobile devices.
3. Google's live streaming infrastructure seems able to scale to whatever capacity is needed.
4. The recording of the stream is immediately available after the event in the IETF's YouTube channel.
5. There is no cost for using the service beyond our local costs to produce the content (and no infrastructure that the IETF itself has to maintain).
Of these, really the most critical reason for using YouTube live streaming is the first - that it streams out over IPv6.
The IETF is the organization behind the IPv6 specification and has declared that all new IETF standards need to incorporate IPv6. Therefore in the spirit of "eating your own dog food" the IETF tries to use services that work over IPv6 whenever possible. Other live video streaming services have met the reasons 2-5 above, but not the #1 reason of working over IPv6.
We have specifically been using what Google used to call "YouTube Live" but now seems to just be calling "YouTube live events" versus Google's newer "Hangouts On Air (HOA)". These YouTube live events are events you schedule in advance and can use with advanced video encoders. An advantage is that these events provide streaming configuration info that I can provide in advance to the company running the audio and video at the event so that they can be prepared in advance. YouTube also helpfully provides a countdown timer for people visiting the event page. We haven't switched to using HOAs because they haven't yet provided the advance configuration information we want.
Anyway it has all worked well for live streaming out plenary sessions for a couple of years now.
Google Doesn't Live Stream Into Germany
However, as we discovered again that week.... Google will not stream live video into Germany! It seems Google has a legal dispute with a German intellectual property rights organization (GEMA) and Google has decided that rather than run into trouble with GEMA they will simply NOT allow live streaming into Germany.
So, alerted to this issue by some IETF remote participants in Germany who were unable to watch the live video streams of the technical plenary earlier in the week, we had arranged to also stream this Thursday session out over the Internet Society's Livestream.com account. Now, unfortunately it would not be available over IPv6 because Livestream.com still only works on legacy IPv4 networks, but Livestream.com did not have the streaming restrictions Google had and so at least people could view the stream in Germany. As a bonus, all the "subscribers" to the Internet Society's Livestream.com channel would also get notified and potentially be able to watch the stream - but the primary reason was so that people in Germany could watch the video stream.
The great thing about IETF meetings is that a massive amount of Internet connectivity is brought into the meeting hotel (because you have 1,200+ engineers who do most of their work across the Internet!) and so there are NO bandwidth problems for streaming. We could probably stream out to a dozen different live streaming services simultaneously if we set up our local software/equipment to do so.
Making The Alternate Stream The Primary Stream
The good news for us was that this "alternative" live video stream set up purely for viewers in Germany could now become the primary video stream. I rapidly updated the Google+ "event page" for this session to note the new URL for streaming and we spread the word through IETF social media channels and email lists. It wasn't 100% seamless but we were able to get people watching the live video stream.
We were also able to direct people to some of the other IETF remote meeting participation mechanisms, including audio streaming and a conferencing system called "Meetecho" that streamed the slides and lower webcam-quality video.
Throughout the hour-long event I kept checking the Live Control Room inside of YouTube to see if we could start the original stream, but we were never able to do so. A couple of times the red warning box went away, but we could not establish a connection from YouTube's streaming service to our equipment on the ground there in Toronto. Finally, as the time went on it became clear that the connection wasn't going to happen and so I just gave up trying.
The good news is that the producer was also making a local copy of the stream that we would be able to upload later to the IETF's YouTube channel.
Lessons Learned
I took away from this experience three primary lessons for all future live streaming sessions. Do note, too, that I think of these as generic lessons for all live streaming services and events. It happens that this time the failure was with Google's YouTube live events service, but the failure could have been with Livestream.com, Google's Hangouts On Air, Ustream or any of the many other live video streaming services out there.
1. Always Promote An Event Page Separate From The Streaming Service
We were able to rapidly redirect people to the new location of the live video stream in large part because we had been promoting the Google+ event page as the place to go to watch the live stream. We had promoted this on the IETF's Twitter account, Facebook page, Google+ page and also over various IETF email lists and on various other websites. All the promotion pointed people to this page.
So the good news was that all we had to do was update this page with the new info and people could switch over to watch the new stream.
We had NOT been promoting the direct YouTube link for the stream. Had we done so, we could have still updated the page through editing the description of the YouTube video and/or leaving comments - but it would not have necessarily been as easy for visitors to see.
Promoting a separate page was a deliberate choice I made based on some previous bad experiences with live streaming where I had to stop a streaming session and restart with a brand new URL. For that reason I've been promoting a separate page.
In fact, for the IETF Plenary sessions, we've been promoting a separate page under IETF control on the IETF website - http://www.ietf.org/live/ - where we can embed the live stream video and also keep the page updated. At the IETF meeting it is possible for me or someone else to easily go in and update that page. Plus it is a very simple URL that we can promote widely.
I don't honestly remember why we didn't use the www.ietf.org/live/ page to stream out this Thursday morning sponsor presentation other than that the decision to live stream the session happened the day before and for whatever reason we went with a Google+ Event page as the page to promote.
Next time we'll probably promote the www.ietf.org/live/ page.
The key point is that you have a page separate from the live streaming service where you can post updates.
2. Have An Alternate Live Stream Either Active Or Ready To Go
As I mentioned previously, in this case we happened to be set up with a second live stream out through Livestream.com purely because we wanted to test the live streaming into Germany. Had remote IETF participants in Germany not asked about this after being unable to view the earlier technical plenary, we wouldn't have had this second stream active.
Next time, we will have a second live streaming service either active or at least on standby ready to go.
At the IETF meetings, we have the luxury of having an insane amount of bandwidth and so there are not the typical connectivity constraints you find in meeting venues. The software and equipment our producer was using could go out to multiple live streaming services. There is really no reason we can't run multiple streams.
For the IETF we still have the IPv6 requirement, which unfortunately Livestream.com does not yet meet. However, it occurred to us after the session that we could have streamed to a Google+ Hangout On Air (HOA) as that would have also streamed out over IPv6 in addition to IPv4. Of course, that would mean relying on two Google services and so you run the risk of having the technical issues affecting one live streaming service also affecting the other - plus there was the whole "streaming into Germany" thing.
We'll definitely keep investigating what other live streaming services may work over IPv6. There are a good number of live video streaming services out there and the number seems to be growing. The company producing the video stream for us also had their own streaming server that we might be able to use as a backup, too. And, yes, we can also have an IPv4-only streaming service available if everything else falls through.
Now, in non-IETF environments where I do have to worry about bandwidth constraints, I will at least have a plan for how I can rapidly spin up a second stream if the first one fails. That's really the key point. What is Plan B and how fast can you make it happen?
3. Have Access To Relevant Social Media Accounts And Other Methods Of Letting People Know
This is perhaps a subset of Lesson #1, but another critical part of our success in redirecting people to the second live stream was that we had access to the relevant social media accounts and other means of spreading the word. I had access to the IETF Google+ page and could make the updates there. Someone else was able to send out a tweet with the new link to the live stream. An email was sent out to all attendees and to other relevant email lists letting them know about the link.
The key point is that when we updated the event page with the new information, we could let people know!
In The End...
... the session was streamed live across the Internet. It was recorded and made available for later viewing. And... we learned a few lessons to make sure our live streaming infrastructure is more resilient next time so that this potential "nightmare" becomes nothing more than just a minor bump and redirection.
What about you? If you do live video streaming what steps have you taken to ensure you can keep streaming in cases like this?
I also recorded an audio commentary about this situation:
If you found this post interesting or useful, please consider either:
- following me on Twitter;
- adding me to a circle on Google+;
- following me on App.net;
- subscribing to my email newsletter; or
- subscribing to the RSS feed.
Sep 02
Watch the 2014 Internet Governance Forum (#IGF2014) Live All This Week from Istanbul (Featured Blog)
The Ninth Annual Internet Governance Forum (IGF) Meeting is taking place all this week, September 2-5, 2014, in Istanbul, Turkey and live video streams are available for all the sessions... I watched the opening session live stream today from my home office in New Hampshire, USA, and the quality was excellent. There was also a live text transcription. More...
