The problem is that while there are tons of guides and gobs of sample code out there, nobody really explains how it works.
The end result is that you program away, and then discover that OOPS! It’s not connecting…
So, let’s see how to make WebRTC actually work!
An Intro to WebRTC
You can also find some code example/demos here.
So why doesn’t your WebRTC code work?
The short answer is that you’re doing it wrong. Don’t worry – it’s not your fault.
The long answer will require some code to look at.
As a good starting point, I suggest this MDN github repo, which I’ll reproduce below in case it disappears:
Note that the above code is just a starting point, and will require some modifications for your particular app!
Okay, here goes…
The actual process of negotiating a WebRTC call is not really clear from the above code.
Naturally, you need a signaling server – in the above code, that’s done via a websocket connection. You can use whatever other messaging system that you want between the 2 clients, but you’ll need something.
The first step is to invite() a remote user to a call.
The invitation involves you calling createPeerConnection(). That just tells your machine where to find the STUN/TURN server for negotiating an actual connection between puters. Your puter then calls getUserMedia() which grabs your cam/mic, and then adds the video/audio tracks to your RTCPeerConnection.
It’s the RTCPeerConnection OFFER that you need to send to your remote client, but hold your horses… Your code shouldn’t actually send any message, or OFFER, to the remote client until RTCPeerConnection.addTrack (or addTransceiver) is called!
When .addTrack() is called, the negotiationneeded even is fired. It’s inside the handleNegotiatioNeeded event handler that you will do createOffer(), setLocalDescription(offer), and then send that OFFER to the remote client.
On the remote client, they will receive the OFFER message via your signaling system, call createPeerConnection() themselves, set their own setLocalDescription(), setRemoteDescription() with your OFFER, and then call createAnswer() and then send that ANSWER back to your client!
This is where it gets hairy… You have to remember that all of this WebRTC stuff is asynchronous. That means that as soon as your client sends an OFFER to the remote party, your browser may also start sending ICE candidates to the remote puter.
Keep in mind that the remote client hasn’t even accepted your OFFER to talk yet!!
So, locally, your client will start firing handleICECandidateEvent several times. That handler will simply forward the ICE candidates to the remote party. The remote party will need to be able to accept those, and call handleNewICECandidateMsg() to do myPeerConnection.addIceCandidate(candidate).
Now, if you’re paying close attention, you should realize two things:
- As soon as you get an OFFER, you MUST create your own myPeerConnection even if you don’t intend to accept the remote call! Otherwise, you can’t accept ICE candidates, and the call won’t work.
- After the remote party sends its ANSWER back to the caller, that does not mean the call is starting! It just means that the remote party will also start sending ICE candidates back to the caller.
In other words, createAnswer() isn’t about answering a video call; it’s about responding to the OFFER with a “COUNTER-OFFER” and saying, “Yeah, I’m here, let’s start negotiating… but my user hasn’t yet decided to accept your call!”
As such, in my own code, the caller sends an OFFER. The remote client receives it, and sends a COUNTEROFFER (the createAnswer).
As the caller, I see my own cam view since my media is already attached to myPeerConnection. The remote party could be viewing that media since the (muted) stream is already being sent to him, but his client JS doesn’t load it until he clicks the “Answer Call” button. And as the caller, my client JS doesn’t unmute the video/audio until the remote client actually accepts my call.
And since the remote guy hasn’t yet clicked the “Answer Call” button, his client JS also hasn’t called .addTrack yet!
But in the meantime, everything has been negotiated in the background so that if he DOES actually answer the call, all is well.
Okay, one more thing: What happens when the remote party accepts your call, and clicks Answer Call??
Easy! The remote client then gets his media, and addTrack‘s or addTransceiver‘s it to his myPeerConnection. When that happens, generally new ICE candidates are sent. But since everything is all set up already, the re-negotiation happens in the blink of an eye…
And POOF! Both sides are connected to the other, my side unmutes the video, and both sides have remote audio/video streams which they display to their user.
Wow, that was crazy
Since everything is done via Promises or async/await, and since the browser will just sort of ‘do it’s own thing’ in terms of ICE candidates and such, your code needs to compensate for all of that.
I can’t count the number of posts I read where people were having exactly the above problems. Hopefully this explanation will make things a bit more clear.
There are other ways of doing it, but I liked this way because there is a minimal amount of mojinations that must occur when the callee clicks the Answer Call button. I prefer as much mojo as possible to happen in the background so that answering a call feels (and is) very speedy.
Also keep in mind that whatever signaling system you’re using, it’s probably also asynchronous. So you really can’t rely on anything happening in a certain order.
If you’re stuck, try adding tons of comments as in the code sample above. That was my savior, because it allowed me to see the ICE candidate timing issue. You only have a short amount of time to accept and addIceCandidate() on both sides. If that fails, the browser just closes the connection.
As long as you think of it as: OFFER, COUNTEROFFER, always createPeerConnection and accept ICE candidates as soon as possible, and then actually ANSWER the call using your own mechanism, everything should work okay.
Well, I hope that prevents somebody else from pulling all their hair out!
I’m off to the wig shop. 😉