Building a Live Streaming Application using OpenCV Library

In this article, I'll walk you through How to Create a live streaming app using OpenCV and Socket Programming with an easy step-by-step tutorial.

Before we start you need to understand a little bit about the OpenCV library and Socket Programming

OpenCV Library

OpenCV-Python is a library of Python bindings designed to solve computer vision problems.

Python is a general-purpose programming language started by Guido van Rossum that became very popular very quickly, mainly because of its simplicity and code readability. It enables the programmer to express ideas in fewer lines of code without reducing readability.

OpenCV-Python makes use of Numpy, which is a highly optimized library for numerical operations with a MATLAB-style syntax. All the OpenCV array structures are converted to and from Numpy arrays. This also makes it easier to integrate with other libraries that use Numpy such as SciPy and Matplotlib.

Socket Programming

Socket programming is a way of connecting two nodes on a network to communicate with each other. One socket(node) listens on a particular port at an IP, while the other socket reaches out to the other to form a connection. The server forms the listener socket while the client reaches out to the server.
They are the real backbones behind web browsing. In simpler terms, there is a server and a client.

Problem Statement:

• 📌 Create Live Streaming Video Chat App without voice using cv2 module of Python

Note: For this Problem Statement, we need two OS (if possible).

Prerequisite:

• Python and Necessary libraries (Numpy and OpenCV) should be installed.

How it works?

Server.py file

Here we are going to write the python code for the live streaming Server Side code.

Step 1:

First of all, We have to import Some Python libraries to make our streaming app those Libraries are cv2, socket, NumPy, struct, and pickle.

import socket, cv2, numpy, pickle, struct

Step 2:

This is the part where we are creating a network socket using the python socket library to make our work easier.

• We just need to specify some parameters inside predefined functions. Our stream app will use UDP protocol
• UDP is commonly used for applications that are “lossy” (can handle some packet loss), such as streaming audio and video. It is also used for query-response applications, such as DNS queries.
• That’s why I used a socket.sock_DGRAM parameter as it means we need UDP protocol.
• After this, we are required to give the IP address and Port Number on which we need our Chat App to make a Connection. Here I have used My Windows Private IP because I am testing it locally
• At last, we need to use the bind function to bind our IP address and Port. So a Connection can be established
• The bind() method is used when a socket needs to be made a server socket. As server programs listen on published ports, it is required that a port and the IP address be assigned explicitly to a server socket. For client programs, it is not required to bind the socket explicitly to a port.

#Creating the Socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#host_name = socket.gethostname()
#host_ip = socket.gethostbyname(host_name)
host_ip = “”
print(‘HOST IP:’, host_ip)
port = 2222
socket_address = (host_ip, port)
print(“Socket created successfully !!”)
server_socket.bind(socket_address)

Step 3:

In this step, we write receiving part

• It says that we have to receive data from the client and get the basic information like the data and IP address of the client.
• The data received from the client will be a NumPy array encoded to streaming data within the client-side. The received data will be decoded.
• imdecode() function reads data from specified memory cache and converts (decodes) data into image format; it is mainly used to recover images from network transmission data.
• Then the data is decoded and the Screen is Show with the Webcam on and If we press Enter Key the program is terminated.

#Socket Listenserver_socket.listen(5)
print("LISTENING AT:", socket_address)
while True:   
    client_socket, addr = server_socket.accept()
    print('Got Connection From:', addr)    
    if client_socket:        
        vid = cv2.VideoCapture(0)        
        while(vid.isOpened()):            
        img, frame = vid.read()            
        a = pickle.dumps(frame)
        message = struct.pack("Q",len(a))+a                 client_socket.sendall(message)            
cv2.imshow('TRANSMITTING VIDEO', frame)            
key = cv2.waitKey(1) & 0xFF            
if key == ord('q'):                
client_socket.close()

Client.py file

Here we are going to write the python code for the live streaming Client Side code.

Step 1:

In this step, we have to import some Python libraries like cv2, socket, pickle and struct

import socket, cv2, numpy, pickle, struct

Step 2:

• Here, In the first line, we are selecting the webcam to use for the video chat
• Then we are using a while loop that will capture our video feed
• Then imshow function is used to display the webcam window
• The data is encoded using imencode
• cv2. imencode() function converts (encode) the image format into streaming data and assigns it to the memory cache. It is mainly used for compressing image data format to facilitate network transmission.
• The encoded data is dumped using the pickle library
• sendto() function is used to send a message on a socket (the encoded data). It shall send a message through a connection-mode or connectionless-mode socket.
• If the Enter Key is Pressed then the program is terminated

#socket create
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host_ip = input("Enter the Server IP: ")
port = 2222     
client_socket.connect((host_ip,port))
data = b""
payload_size = struct.calcsize("Q")
while True:    
    while len(data) < payload_size:        
        packet = client_socket.recv(4*1024)         
        if not packet: 
        break        
        data+=packet    
    packed_msg_size = data[:payload_size]    
    data = data[payload_size:]    
    msg_size = struct.unpack("Q", packed_msg_size)[0]            
    while len(data) < msg_size:        
        data += client_socket.recv(4*1024)    
    frame_data = data[:msg_size]    
    data = data[msg_size:]    
    frame = pickle.loads(frame_data)    
    cv2.imshow("Recived", frame)    
    key = cv2.waitKey(1) & 0xFF    
    if key == ord('q'):       
        break
client_socket.close()

For Demo clone, from the below GitHub link and run it.

https://github.com/ajmalmohammedn/video_chat_application

Thanks, everyone!