Introduce how to set up Charles Proxy on MAC, Windows and mobile phone if experience SSL certificate download failed due to network failure. Share how to use Regular Expression (regex) to rewrite rule in Charles Proxy with debugging.
Without installing SSL certificate on
IOS device for Charles, you only can see the traffic over HTTP. In order
to see the traffic over HTTPS and solve the SSL refusing handshaking
issue, you have to install SSL certificate on IOS device. Install SSL
certificate for IOS device is very simple. Just open Safari from IOS
device, then go to Charlesproxy.com/getssl to download SSL certificate
should be fine. But make sure Charles already has enabled for SSL
Proxying option for all hosts and port number, or at least for the host
that you wanted.
Check this video, it shows you how to install SSL certificate on iPhone. It would be the same way for iPad.
After SSL certificate installed, if you
still cannot see the traffic over HTTPS on your iPhone or iPad, please
make sure device has turned on for trusting this certificate from About
page in Settings as screenshot below:
There are many ways to connect Charles
proxy from IOS device. Here introduce 2 methods. Both 2 methods need
device and Charles connected to the same network.
Method 1: IOS device connects to Charles by computer Ethernet shared hotspot
In this method, first you need to set
up a hotspot from you computer either on Windows or MAC where Charles
was installed on it. Then let IOS device iPhone or iPad connect to the
hotspot. Once connected, configure proxy and port number on IOS device.
Then you are done.
We created this video as an example for how an iPhone connect to Charles by computer Ethernet shared a hotspot on Mac.
Method 2: IOS device connects to Charles by both connecting to the same wifi
If you have a problem to set up a hot
spot from your computer either on Windows or MAC, you can choose this
way to set up your IOS device. It just simple let your device and
Charles installed computer connect to the same wifi, then configure on
your IOS device. Check this video, it tells how to make it work by
connect to same wifi.
Once
your iPhone or iPad set up a connection with Charles proxy by above
either way. You can see all traffic over HTTP. If you find you cannot
see the traffic over HTTPS or tells you Handshake issue, then it needs
to install SSL certificate for Charles proxy. For how to install SSL
certificate on Charles proxy, check the other article How to install SSL certificate on IOS device for Charles proxy.
Here we record several videos shows how to set up a proxy in browsers across platforms on Windows and MAC.
Network proxy can be configured in browser Settings for Charles proxy.
This includes browser FireFox, Chrome, Safari, IE. And the way for the
configuration are similar across Windows and MAC. Just simple set up a
proxy in FireFox or Chrome Settings and let it point to local IP of
machine with port number. Then Charles proxy can catch all traffic of
browser.
One thing needs to mention, after configure a proxy in browser, don't
forgot to take them off if you turn Charles off. Otherwise, Internet
will stop work because browsers tries to send traffic via a proxy that
you set up in browser but you have closed Charles.
How to configure a proxy in FireFox for Charles proxy in Windows
How to configure a proxy in IE for Charles proxy in Windows
How to configure a proxy in FireFox for Charles proxy in MAC
How to configure a proxy in Chrome and Safari for Charles proxy in MAC
* But just be careful on Mac, once this configuration made on Mac in
System Preference via Chrome, it also changes the proxy settings of Mac
machine with proxy. It means your machine will proxy all Internet
traffic in and out. Then when you turn Charles proxy off, then Internet
will be closed as well. So when you don't use Charles for proxy, don't
forget to change proxy setting back in System Preference. This is
different from when you enable MAC Proxy in Charles proxy. When you enable MAC Proxy
in Charles proxy, turning off Charles proxy will disable proxy function
of Mac machine. But if you made change in System, turning off Charles
proxy, it won't disable proxy function.
Fix "Remote host closed connection during handshake", "You may need to configure your browser or application to trust the Charles Root Certificate"
When you see above failure in Charles Proxy on Mac, it means you need to install Charles SSL certificate in order to check the traffic of browser or application between client and server on Mac. This article will show you how to solve above handshake issue.
Each website or application has their own SSL authentication request, then you can install SSL certificate for each website or application if you want to do that. Every time when you visit a https website via Charles without SSL certificate installed for Charles, it will give you a warning for authentication accessing, you can accept authentication without any problem and go ahead to browse website. But you can avoid those popup warning each time when visit each website or application via Charles. The only thing you need to do is to install Charles Root Certificate for web access on Mac. After install Charles Root Certificate on Mac for web access, you can read most of traffic between client and server end.
Install Charles Root Certificate on Mac for browser, the processing needs you to add Charles certificate in keychains.
Check this video how to install Charles Root Certificate on Mac for Chrome and Safari:
Check this video how to install Charles Root Certificate on Mac for FireFox:
Install Charles Root Certificate will solve most application or website SSL authentication issue because Charles use regular way to encrypt and decrypt message. However for some special websites or applications if they use or define their own specific SSL certificate rather than regular encrypt/decrypt way to handle message, install Charles Root Certificate won't help to solve the problem. You still won't see plain traffic between client and server end and you still see SSL handshake issue in Charles. In this case, you need to request those specific SSL certificate from server owner, and add this specific certificate in the Charles. then you will see the encrypted/decrypted message between client and server end.
To install specific SSL certificate for special website or application, you can go through the following processing: 1. Obtain the specific SSL certificate from Server side owner who issues SSL certificate. 2. Go Charles Proxy > SSL Proxying Settings 3. Select Client Certificates tab 4. Click Add button to add Host, Port, and click Choose button to upload certificate file. Now Specific SSL certificate install successfully.
To install Charles Root Certificate on Mac for browser and application:
1. Open Charles
2. Go to "Proxy" and select "Mac OS X Proxy"
3. Go to "Help" > "SSL Proxying" > "Install Charles Root Certificate.."
4. Select "Charles Proxy Custom Root Certificate" to open Charles Proxy Custom Root Certificate page
5. Expand "Trust"
6. In "When using this certificate", select "always trust". Close this page and give the password for the installation permission.
Fix "No request was made. Possibly the SSL certificate was rejected"
If you see above message "Possibly the SSL certificate was rejected" in Charles proxy, it means you need to install SSL certificate in your Windows machine in order to see the encrypted traffic between IE and server.
When you put Charles in between client and server, you may find some messages exchanged between client and server are readable and some are not readable. Then you have to check if these message are sent out over HTTP or HTTPS. If the message sent between client and server over HTTP instead of HTTPS, it won't see the SSL certificate issue such as "SSL certificate was rejected" because message sent over HTTP without any encryption and no SLL certificate requested in between client and server.
However if request and response sent over HTTPS and without any SSL certificate installed on the machine in right place, you will see the "SSL certificate was rejected". That is why you see some communication between client and server are readable and some are not readable.
For message exchanged over HTTPS, each website accepts it's own SSL authentication certificate. Without Charles setting up in between client and server, client end and server can exchange it's own SSL authentication certificate and communicate without any problem. But when put Charles in between client and server, then communicate becomes client sends request to Charles, and Charles forwards request to server. At this moment, server won't understand Charles forwarded request because of lacking SSL authentication certificate.
To solve this issue, we can install Charles root certificate. After install Charles root certificate, client can communicate with sever via Charles in between.
Check this video, it shows you how to install SSL certificate easily for Charles proxy in Windows for IE and application.
Check this video, it shows you how to install SSL certificate easily for Charles proxy in Windows for FireFox.
Understanding how SSL certificate works would be a big help to understand how HTTP and HTTPs communicate and why it needs to install SSL certificate.
How to set up VPN with Charles Proxy for the phone to see the traffic of the phone
We all know our world is isolated from
one country to the other country from Internet for some geo restrictions
or political restrictions. And with all kinds of different reasons, we
have to break the wall to access other country resource. For example,
you are traveling to other country and you have to access back the
original country resource for the time being, or your company outsource
the jobs and employees have to test the job to see if it is working on
the job requested country, etc.
How to break the wall from Internet
to access other country resource? The good thing is we have VPN which
can help us to solve the problem. There are tons of info on Internet
regarding how to set up VPN on your computer or phone. Here we show you
how to set up VPN and Charles together to observe the traffic between
client and server for testing purpose.
"SSLHandshake: Remote host closed connection during handshake. " "SSLHandshake: Received fatal alert: certificate_unknown"
The reason you see the error "SSLHandshake: Remote host closed connection during handshke." because the request was sent out from web browser over the HTTPS, if all the requests or responses sent out over HTTP, you won't see this error in Charles because all requests sent out over HTTP are plain message without any encrypted. Either request or response sent out over HTTP or HTTPS, it depends on the website itself development system. As we said message over HTTP is plain message without any encrypted, then it will have less security. But if request or response sent out over HTTPS, all message are encrypted and have more security than over HTTP.
All messages exchanged between client and server are encrypted over HTTPS. If you want to see decoded plain text message between client end of phone and server over HTTPS, you have to install SSL certificate for Charles on your phone device, to let Charles translate these encrypted message for you by passing the installed SSL certificate between client and server. Check More details.
How to simulate network failure or server is down by using Charles Proxy Black List
There are multiple ways to simulate network failure or simulate server is down at sever end, then we can check how client app behaves when encounter such issues. Charles Proxy provide such function to help us to simulate network failure and server down situations. We will give an example for how to using this function from Charles.
Here is a
way that you can set up a black host list in Charles Proxy to simulate
network failure or server is unreachable for the specific hosts that you
want to block. Charles gives 2 ways to block the hosts. One way is to drop the connection. And the other way is to return 403 response. And you just need to add a host to the list that you want to block. Go ahead to check the More details.
