In the era of big data, processing large datasets efficiently is crucial. Apache Spark, an open-source distributed computing system, has become a go-to tool for handling massive data workloads. PySpark is its Python API, making it easier for data engineers and analysts to work with Spark using Python.

In this guide, we will explore PySpark’s architecture, key components, and practical commands to help you get started with big data processing.

🔥 What is PySpark?

PySpark is the Python interface for Apache Spark, allowing users to leverage Spark’s powerful distributed computing capabilities using Python. It is widely used for big data analytics, machine learning, and ETL (Extract, Transform, Load) processes.

Why Learn PySpark?

• 🚀 Handles large datasets efficiently
• ⏳ Faster than traditional frameworks like Pandas for big data
• 🔀 Works seamlessly with distributed computing clusters
• ☁️ Supports cloud platforms like Databricks and Google Cloud

🏗 PySpark Architecture

PySpark follows a distributed computing model, breaking large tasks into smaller ones that run in parallel. Its architecture includes:

1️⃣ Driver Program

• The driver is the main control program that runs the SparkContext.

2️⃣ Cluster Manager

• Allocates resources across a cluster (e.g., YARN, Mesos, Kubernetes, or Standalone mode).

3️⃣ Executors

• Run tasks assigned by the driver and store data in memory or disk.

4️⃣ RDDs (Resilient Distributed Datasets)

• The fundamental data structure in Spark, ensuring fault tolerance and parallel processing.

🛠 Setting Up PySpark

You can install PySpark using pip:

pip install pyspark

To start PySpark in an interactive mode:

pyspark

For Jupyter Notebook users:

pip install jupyter findspark

Then, in Python:

import findspark
findspark.init()
import pyspark
from pyspark.sql import SparkSession

spark = SparkSession.builder.appName("MyApp").getOrCreate()
print(spark)

📊 PySpark Components

PySpark consists of several key components:

1️⃣ SparkContext

• The entry point to Spark’s core functionality.

from pyspark import SparkContext
sc = SparkContext("local", "MyApp")
print(sc)

2️⃣ SparkSession

• The unified entry point for working with DataFrames and Datasets.

from pyspark.sql import SparkSession
spark = SparkSession.builder.appName("MyApp").getOrCreate()

3️⃣ RDDs (Resilient Distributed Datasets)

• Immutable, distributed collections of data.

rdd = sc.parallelize([1, 2, 3, 4, 5])
print(rdd.collect())  # [1, 2, 3, 4, 5]

4️⃣ DataFrames

• Similar to Pandas DataFrames but optimized for distributed processing.

data = [("Alice", 25), ("Bob", 30)]
columns = ["Name", "Age"]
df = spark.createDataFrame(data, columns)
df.show()

⚡ Essential PySpark Commands

Example:

df.show()  # Displays the DataFrame

📌 PySpark Cheat Sheet

🎯 Tips to Learn PySpark Faster

• ✅ Start with small datasets before moving to big data.
• ✅ Practice using DataFrames and RDDs in Jupyter Notebook.
• ✅ Explore official documentation and community resources.
• ✅ Work on real-world datasets to solidify concepts.
• ✅ Try cloud-based platforms like Databricks for hands-on experience.

🎯 Conclusion

PySpark is a powerful tool for processing big data at scale. With its support for distributed computing and cloud platforms like Databricks, it is a must-have for any data engineer. This guide covered:

• PySpark architecture & components
• Essential commands & DataFrame operations

Now you’re ready to dive deeper into big data processing with PySpark! 🚀

Happy Pysparking!

Apache Kafka is a distributed event-streaming platform designed for high-throughput, fault-tolerant, real-time data processing. From building event-driven architectures to powering critical business operations, Kafka is a go-to tool for data engineers and developers alike.

In this blog, we will dive deep into Kafka’s architecture, set up a Kafka environment using Docker Compose, and explore commands to manage topics, producers, and consumers.

⚙️ Kafka Architecture Overview

Kafka’s architecture comprises the following components:

1. Producers: Send messages to Kafka topics.
2. Topics: Logical channels where data is organized.
3. Partitions: Subdivisions within a topic that enable parallel processing.
4. Consumers: Read messages from topics.
5. Brokers: Kafka servers that handle message storage and distribution.
6. ZooKeeper: Manages cluster metadata and coordination.

Key Features:

• Distributed and Fault-Tolerant
• Horizontal Scalability
• Real-Time Stream Processing
• Exactly-Once Semantics

🛠 Setting Up Kafka with Docker Compose

To get started with Kafka, we will set up a local Kafka environment using Docker Compose. Below is the docker-compose.yml configuration:

docker-compose.yml
version: '3'
services:
  zookeeper:
    image: zookeeper:3.6.3
    ports:
      - "2181:2181"

  kafka:
    image: wurstmeister/kafka:2.13-2.7.0
    ports:
      - "9092:9092"
    environment:
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    depends_on:
      - zookeeper
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

Steps to Run:

1. Save the above content into a file named docker-compose.yml.
2. Open a terminal and navigate to the directory containing the file.

3. Run the command:

   docker-compose up -d
   

4. Verify that the Kafka and Zookeeper containers are running using:

   docker ps
   

🚀 Hands-On with Kafka Commands

Once your Kafka setup is running, follow these steps to interact with Kafka.

Access Kafka Container

To access the Kafka container, execute:

docker exec -it <kafka_container_id> /bin/bash

Locate Kafka Binaries

Inside the container, navigate to the Kafka binaries:

cd /opt/kafka/bin

Creating a Topic

Create a new topic named test-topic:

./kafka-topics.sh --create --topic test-topic --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1

Running a Kafka Producer

To send messages to the test-topic, run the producer:

./kafka-console-producer.sh --topic test-topic --bootstrap-server localhost:9092

Type messages in the terminal and press Enter to send them.

Running a Kafka Consumer

To consume messages from test-topic:

1. Open a new terminal and access the Kafka container.

2. Navigate to the Kafka binaries:

   cd /opt/kafka/bin
   

3. Run the consumer:

   ./kafka-console-consumer.sh --topic test-topic --bootstrap-server localhost:9092 --from-beginning
   

After that when you send query from producer, it will reflected in consumer terminal quickly.

Creating a Topic with Multiple Partitions

Create a topic named test-topic-two with three partitions:

./kafka-topics.sh --create --topic test-topic-two --bootstrap-server localhost:9092 --replication-factor 1 --partitions 3

If the topic already exists, you will receive an error message.

📊 Deep Dive into Topics and Partitions

What Are Partitions?

Partitions divide a topic into smaller, manageable chunks, enabling parallelism and scalability. Each partition can be processed by a separate consumer.

Key-Value Advantage

Messages can be routed to specific partitions based on keys, ensuring data locality for related events.

🔍 Troubleshooting Tips

Checking Existing Topics

List all topics in the Kafka cluster:

./kafka-topics.sh --list --bootstrap-server localhost:9092

Viewing Topic Details

Describe a topic to inspect its configuration:

./kafka-topics.sh --describe --topic test-topic --bootstrap-server localhost:9092

Debugging Producer and Consumer

• If the producer or consumer cannot connect, verify that the Kafka container is running and reachable on port 9092.

• Check the logs of the Kafka container:

  docker logs <kafka_container_id>
  

🌐 Conclusion

Congratulations! You have successfully set up Kafka, created topics, and interacted with producers and consumers. With this knowledge, you are equipped to dive deeper into advanced Kafka features such as stream processing with Kafka Streams, connecting external systems using Kafka Connect, and deploying Kafka clusters in production.

Happy Streaming!

Apache Airflow is an open-source platform to programmatically author, schedule, and monitor workflows. It enables data engineers and developers to manage workflows efficiently with a rich UI and dynamic pipeline creation using Python.

In this guide, you will learn how to install Airflow using Docker, explore its components and architecture, and create your first DAG (Directed Acyclic Graph) using BashOperator.

⚡ Overview of Apache Airflow

Apache Airflow provides a powerful and flexible way to programmatically orchestrate workflows. Here’s a quick breakdown of its architecture:

Components:

• Scheduler: Monitors tasks and DAGs, triggering task instances once dependencies are met.
• Webserver: Provides a rich user interface to manage workflows.
• Worker: Executes tasks assigned by the scheduler.
• Triggerer: Runs an event loop for deferrable tasks.
• Database: Stores metadata about workflows and tasks.
• Executor: Defines how tasks are executed (e.g., CeleryExecutor for distributed execution).

Architecture Overview:

Airflow uses a centralized metadata database to track workflows, and its components interact through this database. Workflows are defined in Python scripts as DAGs, which describe dependencies and execution order.

🛠️ Installing Apache Airflow Using Docker

Prerequisites

Ensure the following are installed:

• Docker (Installation Guide)
• Docker Compose (Installation Guide)

Fetching docker-compose.yaml

To deploy Airflow using Docker Compose, download the docker-compose.yaml file:

curl -LfO 'https://airflow.apache.org/docs/apache-airflow/2.10.4/docker-compose.yaml'

Important: From July 2023, Compose V1 stopped receiving updates. Upgrade to a newer version of Docker Compose to ensure compatibility.

This file defines several services:

• airflow-scheduler: Monitors and schedules workflows.
• airflow-webserver: Accessible at http://localhost:8080.
• airflow-worker: Executes tasks.
• airflow-triggerer: Manages deferrable tasks.
• airflow-init: Initializes the Airflow environment.
• postgres: Airflow metadata database.
• redis: Message broker between scheduler and worker.

Optionally, enable Flower (a monitoring tool) by running:

docker compose --profile flower up

Flower will be available at http://localhost:5555.

Setting Up the Environment

Step 1: Set the Right Airflow User

On Linux:

mkdir -p ./dags ./logs ./plugins ./config
echo -e "AIRFLOW_UID=$(id -u)" > .env

For other OS, create an .env file manually with:

AIRFLOW_UID=50000

Step 2: Initialize the Database

Run the following to initialize the environment and database:

docker-compose up airflow-init

You should see a message indicating successful initialization:

airflow-init_1       | Upgrades done
airflow-init_1       | Admin user airflow created
airflow-init_1       | 2.10.4
start_airflow-init_1 exited with code 0

Step 3: Start Airflow

Start all services:

docker-compose up

Access the Airflow UI at http://localhost:8080 with the username airflow and password airflow.

Cleaning Up

To clean up the environment:

1. Stop services and remove volumes:

docker-compose down --volumes --remove-orphans

1. Remove the directory containing docker-compose.yaml:

rm -rf '<DIRECTORY>'

1. Re-download docker-compose.yaml and start again.

✍️ Writing Your First DAG

Steps to Create and Run Your First DAG

1. Create a Python file (e.g., first_dag.py) in the ./dags folder with the following content:

from datetime import datetime, timedelta
from airflow import DAG
from airflow.operators.bash import BashOperator

default_args = {
    'owner': 'rahman',
    'retries': 5,
    'retry_delay': timedelta(minutes=2)
}

with DAG(
    dag_id='our_first_dag_v6',
    default_args=default_args,
    description='This is our first dag that we write',
    start_date=datetime(2024, 7, 29, 2),
    schedule_interval='@daily'
) as dag:
    task1 = BashOperator(
        task_id='first_task',
        bash_command="echo hello world, this is the first task!"
    )

    task2 = BashOperator(
        task_id='second_task',
        bash_command="echo hey, I am task2 and will be running after task1!"
    )

    task3 = BashOperator(
        task_id='third_task',
        bash_command="echo hey, I am task3 and will be running after task1 at the same time as task2!"
    )

    task1 >> [task2, task3]

1. Save the file and refresh the Airflow UI (http://localhost:8080).

1. Locate the newly added DAG, enable it, and click on the DAG to view its details.

1. Trigger the DAG run manually by clicking the play button.

2. Monitor the job status in the UI. You can view logs for each task by clicking on it in the Graph View or Tree View.

1. Verify the job completion and review logs for insights.

With these steps, you have successfully installed Apache Airflow using Docker, created a DAG, and monitored its execution. Happy workflow orchestration!

Tuning a foundation model is a crucial step in customizing AI systems to generate desired outputs efficiently. This guide provides an in-depth walkthrough of how to perform prompt tuning for foundation models using Watsonx.ai, including setting up a tuning experiment and optimizing for specific tasks.

Requirements

Before diving into tuning, ensure you have access to the Projects in Watsonx.ai. Note that availability varies by plan and data center. Confirm the foundation models available for tuning in your region.

To begin, a default project named sandbox is created for Watsonx.ai users. If you don’t see this project, create one manually by following these steps:

1. Expand Projects from the main menu and click All projects.
2. Click New project.
3. Name your project and optionally add a description.
4. Click Create.

For additional project options like reporting or logging, refer to the Creating a project

🛠️ Before You Begin

Make decisions about the following tuning options:

1. Select the Foundation Model:
   • Choose a model that aligns with your use case.
2. Prepare Example Prompts:
   • Create example prompts based on your prompt engineering work.

Download the datasets from Hugging face

1. Dataset Details:

In this Project I am using the alespalla/chatbot_instructions_prompts

1. Download and clean the dataset:

Download the dataset and change into json or jsonl format (eg.input-output fromat)

1. Upload the dataset to IBM cos or You can Upload it manually

🔧 How to Tune a Foundation Model

Step 1: Start a Tuning Experiment

1. From the Watsonx.ai home page, select your project.
2. Click New asset > Tune a foundation model with labeled data.
3. Name your tuning experiment and optionally add a description and tags.
4. Click Create.

Step 2: Choose a Foundation Model

1. Click Select a foundation model.
2. Browse through available models and view detailed information by selecting their tiles.
3. Once decided, click Select.

Step 3: Upload the dataset and Initialize the Prompt

Upload the dataset from cos or manually

Choose one of the following initialization options:

• Text: Provide specific initialization text.
• Random: Let the system generate initialization values.

Adding Initialization Text

For the Text method, provide task-specific instructions:

• Classification: Include task details and class labels, e.g., Classify sentiment as Positive or Negative.
• Generation: Provide a detailed request, e.g., Generate an email promoting remote work.
• Summarization: Specify objectives, e.g., Summarize meeting highlights.

Step 4: Specify the Task Type

Select the task type that fits your goal:

• Classification: Assign categorical labels.
• Generation: Produce text outputs.
• Summarization: Extract main ideas from text.

Step 5: Add Training Data

1. Upload training data or use an existing project asset.
2. To preview data format templates, expand What should your data look like?
3. Optionally, adjust the maximum tokens allowed for input and output to optimize processing time.

Step 6: Configure Tuning Parameters

1. Edit parameter values for the tuning experiment by clicking Configure parameters.
2. After adjustments, click Save.

Step 7: Start Tuning

Click Start tuning to begin the experiment. The duration depends on the dataset size and compute resource availability. Once complete, the status will display as Completed.

🧩 Tips for Token Management

Tokens are key units for natural language processing. Adjust token limits to optimize performance:

• Maximum Input Tokens: Controls input size (e.g., 256 tokens).
• Maximum Output Tokens: Limits generated output size (e.g., 128 tokens).

Example

For classification tasks, reduce output size to encourage concise results (e.g., Positive or Negative). —

📊 Evaluating the Tuning Experiment

The experiment results include a loss function graph that visualizes model improvement:

• X-axis: Epochs.
• Y-axis: Difference between predicted and actual results.

  🔧 Deploy a Tuned Model

Steps to Deploy:

1. From the navigation menu, expand Projects and click All projects.
2. Select your project and navigate to the Assets tab.
3. Open the tuning experiment associated with the model you wish to deploy.
4. From the Tuned models list, locate the completed experiment and click New deployment.

1. Provide a name for the tuned model. Optionally, add a description and tags.
2. Select a deployment container:
   • This project: For testing within the project.
   • Deployment space: For production-ready deployment.

Click Deploy to complete the process.

🔍 Testing the Deployed Model

Test Options:

• Project: Ideal for development and testing phases.
• Deployment space: Test programmatically or via the API Reference tab.
• Prompt Lab: Offers an intuitive interface for detailed prompting and testing.

Testing in Prompt Lab:

1. Open the deployed model in the project or deployment space.
2. Click Open in Prompt Lab.
3. Input a prompt tailored to the model’s tuning and click Generate.

Trained Model response

This is the output generated from the trained granite model

Without Trained Model response

This the normal output from the untrained model

Conclusion

Tuning foundation models allows for customization that aligns AI outputs with your specific needs. By following this guide, you’ll maximize the potential of Watsonx.ai for your use cases.

IBM’s watsonx.ai provides a flexible and robust platform for deploying foundation models, enabling developers to integrate them into generative AI solutions. In this guide, we will show you how to deploy the Falcon-7B foundation model from Hugging Face on IBM Cloud using Windows commands.

You will learn about the necessary steps to:

1. Ensure the model meets the requirements for deployment.
2. Download and prepare the model in the correct format.
3. Upload it to IBM Cloud Object Storage.
4. Deploy and test the model using watsonx.ai.

📋 Prerequisites

Before starting, ensure you have:

• An IBM Cloud account.
• Access to watsonx.ai (trial or paid).
• A Hugging Face account with API token.
• A Windows system with Python and necessary tools installed.

🛠️ Step-by-Step Deployment Guide

Step 1: Ensure Model Compatibility

Your foundation model must meet the following criteria to be deployed on watsonx.ai:

• Compatible with the Text Generation Inference (TGI) standard.
• Built with supported model architecture and gptq model type.
• Available in safetensors format.
• Includes config.json and tokenizer.json files.

💡 Tip: You can verify these files exist for the Falcon-7B model on Hugging Face.

Step 2: Download the Model

To download the Falcon-7B model on Windows:

# Set up a virtual environment
python -m venv myenv
myenv\Scripts\activate

# Install Hugging Face CLI
pip install -U "huggingface_hub[cli]"

# Log in to Hugging Face
huggingface-cli login --token YOUR_HF_TOKEN

# Set model and directory variables
set MODEL_NAME=tiiuae/falcon-7b
set MODEL_DIR=C:\models\falcon-7b
mkdir %MODEL_DIR%

# Download the model
huggingface-cli download %MODEL_NAME% --local-dir %MODEL_DIR% --cache-dir %MODEL_DIR%

(Optional) Step 3: Convert the Model

Convert the model to meet TGI requirements for Text Generation:

# Pull the TGI image
docker pull quay.io/modh/text-generation-inference:rhoai-2.8-58cac74

# Convert the model
docker run --rm -v %MODEL_DIR%:/tmp quay.io/modh/text-generation-inference:rhoai-2.8-58cac74 bash -c "export MODEL_PATH=/tmp; text-generation-server convert-to-safetensors ${MODEL_PATH}; text-generation-server convert-to-fast-tokenizer ${MODEL_PATH}"

Step 4: Upload to Cloud Object Storage

Prepare and upload the model to IBM Cloud Object Storage:

# Install AWS CLI
pip install awscli

# Set environment variables
set AWS_ACCESS_KEY_ID=<your_aws_access_key>
set AWS_SECRET_ACCESS_KEY=<your_aws_secret_access_key>
set ENDPOINT=<s3_endpoint_url>
set BUCKET_NAME=<bucket_name>
set MODEL_FOLDER=<model_folder>

# Upload the model
aws --endpoint-url %ENDPOINT% s3 cp %MODEL_DIR% s3://%BUCKET_NAME%/%MODEL_FOLDER%/ --recursive --follow-symlinks

Step 5: Import the Model to watsonx.ai

1. Navigate to your deployment space in watsonx.ai.
2. Go to Assets → Import.
3. Choose Custom Foundation Model.
4. Connect to your cloud storage and select the folder containing the model.

Step 6: Deploy the Model

Then create a space and deploy the model and wait for teh status from initializing to Deployed.

After the Green Light you are ready to go..

Step 7: Test Your Deployment

Use the watsonx.ai Prompt Lab or API to test your model:

curl -X POST "https://<your_cloud_hostname>/ml/v1/deployments/<deployment_id>/text/generation?version=2024-01-29" ^
-H "Authorization: Bearer <your_token>" ^
-H "Content-Type: application/json" ^
--data '{
 "input": "Hello, what is your name?",
 "parameters": {
    "max_new_tokens": 200,
    "min_new_tokens": 20
 }
}'

Test Falcon-7B-model in prompt lab Adjust the tokens, decoding method and system prompt to get your answers right.

🎉 Summary

By following this guide, you have successfully deployed Falcon-7B to IBM Cloud using watsonx.ai. You can now leverage this powerful foundation model for generative AI applications tailored to your needs.

For more advanced features and configurations, visit the IBM watsonx.ai documentation.

View more blogs at my https://abdulrahmanh.com/blog

Happy Deploying!

Artificial Intelligence (AI) is revolutionizing industries by enabling machines to perform tasks that traditionally required human intelligence. IBM’s Watsonx.ai is at the forefront of this transformation, offering tools to build, fine-tune, and deploy AI models with ease. In this blog, we’ll explore foundational AI concepts, generative AI capabilities, and how to get started with Watsonx.ai.

⚡ Key AI Terms and Definitions

1. Artificial Intelligence

The simulation of human intelligence by machines, enabling them to perform tasks like reasoning, learning, and problem-solving.

2. Machine Learning

A subset of AI focused on developing algorithms that allow computers to learn from data and make decisions based on statistical predictions.

3. Deep Learning

A subset of machine learning that uses artificial neural networks with multiple layers to process vast amounts of data. It excels at handling unstructured data like images and text.

4. Foundation Models

Specific types of deep learning models built using neural network architectures like transformers. These models are pre-trained on vast datasets and fine-tuned for specific tasks.

5. Generative AI

AI algorithms capable of creating new content such as text, images, code, or audio. Unlike traditional AI, generative AI generates outputs rather than simply recognizing patterns.

6. Large Language Models (LLMs)

A type of foundation model trained on extensive text datasets using self-supervised learning. LLMs can perform tasks ranging from natural language understanding to code generation.

7. Hallucination

A phenomenon in LLMs where the system generates incorrect or nonsensical outputs that may appear plausible.

8. Natural Language Processing (NLP)

Technology that enables computers to understand, interpret, and generate human language in text or spoken forms.

9. Prompt

The input or query used to interface with AI models. Well-crafted prompts can improve the accuracy and relevance of AI responses.

10. Prompt Engineering

The process of designing effective prompts to optimize the performance of AI models.

11. Decoder-only Model

Models designed specifically for generative AI tasks, such as GPT-based architectures.

12. Encoder-only Model

Models optimized for non-generative tasks, such as text classification or sentiment analysis.

13. Encoder-Decoder Model

Models that combine encoding and decoding mechanisms, supporting both generative and non-generative tasks efficiently.

14. Tokens

Units of text (e.g., words, subwords, or characters) used by AI models. Tokenization is the process of converting text into these units for model processing.

🛠️ Pre-trained Models

Pre-trained models are AI models that have already been trained on large datasets to perform general tasks. These models can be fine-tuned for specific use cases, saving time and computational resources.

Benefits of Pre-trained Models:

1. Faster Development: Avoid starting from scratch.
2. Cost-Effective: Reduce training costs by leveraging existing models.
3. High Accuracy: Benefit from the vast amount of data and computational power used during pre-training.

Examples of Pre-trained Models in Watsonx.ai:

1. Text Models: LLMs for tasks like summarization, translation, and content generation.
2. Image Models: Models trained for image recognition and object detection.
3. Code Models: Models optimized for generating and debugging code.

🚀 Getting Started with Watsonx.ai

Watsonx.ai provides an intuitive platform to explore and deploy foundation models. Here’s how you can get started:

Step 1: Access Watsonx.ai

• Sign up for IBM Cloud and navigate to the Watsonx.ai section.
• Log in with your IBM account to access the platform.

Step 2: Explore Foundation Models

• Browse the library of pre-trained models.
• Select a model suited to your task, such as text generation or image classification.

Step 3: Fine-tune Models

• Use your own dataset to fine-tune pre-trained models for specific use cases.
• Adjust parameters like temperature and max tokens for optimal performance.

Step 4: Deploy Models

• Deploy your model as an API for integration into applications.
• Use the Watsonx.ai SDK for seamless interaction with your deployed models.

🌐 Conclusion

Watsonx.ai empowers businesses and developers to harness the potential of AI with minimal effort. From understanding foundational concepts to deploying state-of-the-art models, Watsonx.ai provides the tools needed to succeed in the AI era.

Take the first step today and explore the capabilities of Watsonx.ai. The future of AI is here!

Welcome to the future of AI-powered image analysis! IBM has recently released advanced vision models as part of their Watson AI suite. These models combine cutting-edge image recognition with conversational AI capabilities, allowing users to analyze images and ask questions about their contents seamlessly.

In this blog, we’ll explore how to build a simple Streamlit application to interact with these models. Whether you’re analyzing photos, diagrams, or charts, this app can provide insights and context through intelligent conversations.

🎯 What Makes IBM’s Vision Models Unique?

IBM’s latest vision models, like the Meta-LLaMA 3-2-90b Vision Instruct, are designed to:

1. Analyze Visual Data: Extract meaningful insights from images, from object recognition to contextual understanding.
2. Enable Conversational Interaction: Use natural language to query images, blending vision and language capabilities.
3. Empower Developers: Simplify integration into apps using intuitive APIs.

Key Features:

• Seamless integration with Streamlit for rapid prototyping.
• High accuracy in visual and conversational tasks.
• Easy-to-use Watson API for streamlined development.

🚀 Building the Chat with Images App

Let’s dive into creating a Streamlit app that connects with IBM’s vision models. This app allows users to upload an image, analyze it using the Watson AI model, and interact with it via chat.

Prerequisites

1. An IBM Cloud Account with access to Watson AI services.

2. Python installed on your machine.

🛠️ Code Implementation

Part 1: Setup and Environment Configuration

This section initializes necessary libraries, loads environment variables, and provides utility functions for image conversion and authentication.

import streamlit as st
import base64
from PIL import Image
import os
from dotenv import load_dotenv
import requests

# Load environment variables
load_dotenv()
api_key = os.getenv("IBM_API_KEY")

def convert_image_to_base64(uploaded_file):
    """Convert uploaded image to Base64 format."""
    bytes_data = uploaded_file.getvalue()
    base64_image = base64.b64encode(bytes_data).decode()
    return base64_image

def get_auth_token(api_key):
    """Retrieve authentication token using IBM API key."""
    auth_url = "https://iam.cloud.ibm.com/identity/token"

    headers = {
        "Content-Type": "application/x-www-form-urlencoded",
        "Accept": "application/json"
    }

    data = {
        "grant_type": "urn:ibm:params:oauth:grant-type:apikey",
        "apikey": api_key
    }

    response = requests.post(auth_url, headers=headers, data=data, verify=False)

    if response.status_code == 200:
        return response.json().get("access_token")
    else:
        raise Exception("Failed to get authentication token")

Part 2: User Interaction and State Management

Here, we define the app’s main logic, including file upload handling, chat state initialization, and chat message rendering.

def main():
    st.title("Chat with Images")

    # Initialize chat history and uploaded file state
    if "messages" not in st.session_state:
        st.session_state.messages = []
    if "uploaded_file" not in st.session_state:
        st.session_state.uploaded_file = None

    # Clear uploaded image button
    if st.session_state.uploaded_file:
        if st.button("Clear Uploaded Image"):
            st.session_state.uploaded_file = None
            st.session_state.messages = []

    # User input: Upload an image
    uploaded_file = st.file_uploader("Choose an image...", type=["jpg", "jpeg", "png"])
    if uploaded_file is not None:
        st.session_state.uploaded_file = uploaded_file
        image = Image.open(uploaded_file)
        with st.chat_message("user"):
            st.image(image, caption='Uploaded Image', use_container_width=True)
            base64_image = convert_image_to_base64(uploaded_file)
            st.session_state.messages.append({"role": "user", "content": [{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{base64_image}"}}]})

    # Display chat messages
    for msg in st.session_state.messages[1:]:
        if msg['role'] == "user":
            with st.chat_message("user"):
                if msg['content'][0]['type'] == "text":
                    st.write(msg['content'][0]['text'])
        else:
            st.chat_message("assistant").write(msg["content"])

Part 3: API Integration and Response Handling

This part handles API requests to Watson’s model, processes the responses, and updates the chat interface.

    # User input: Chat message
    user_input = st.chat_input("Type your message here...")

    if user_input:
        message = {"role": "user", "content": [{"type": "text", "text": user_input}]}
        st.session_state.messages.append(message)
        st.chat_message(message['role']).write(user_input)

        # Prepare and send API request
        url = "https://us-south.ml.cloud.ibm.com/ml/v1/text/chat?version=2023-05-29"

        model_messages = []
        latest_image_url = None
        for msg in st.session_state.messages:
            if msg["role"] == "user" and isinstance(msg["content"], list):
                content = []
                for item in msg["content"]:
                    if item["type"] == "text":
                        content.append(item)
                    elif item["type"] == "image_url":
                        latest_image_url = item
                if latest_image_url:
                    content.append(latest_image_url)
                model_messages.append({"role": msg["role"], "content": content})
            else:
                model_messages.append({"role": msg["role"], "content": [{"type": "text", "text": msg["content"]}] if isinstance(msg["content"], str) else msg["content"]})

        body = {
            "messages": [model_messages[-1]],
            "project_id": "833c9053-ef07-455e-819f-6557dea2f8bc",
            "model_id": "meta-llama/llama-3-2-90b-vision-instruct",
            "decoding_method": "greedy",
            "repetition_penalty": 1,
            "max_tokens": 900
        }

        try:
            YOUR_ACCESS_TOKEN = get_auth_token(api_key)

            headers = {
                "Accept": "application/json",
                "Content-Type": "application/json",
                "Authorization": f"Bearer {YOUR_ACCESS_TOKEN}"
            }

            response = requests.post(
                url,
                headers=headers,
                json=body
            )

            if response.status_code == 200:
                res_content = response.json()['choices'][0]['message']['content']
                if isinstance(res_content, list):
                    res_content = " ".join([item.get("text", "") for item in res_content])
                st.session_state.messages.append({"role": "assistant", "content": res_content})
                with st.chat_message("assistant"):
                    st.write(res_content)
            else:
                error_message = "Sorry, I couldn't process your request. Please try again later."
                st.session_state.messages.append({"role": "assistant", "content": error_message})
                with st.chat_message("assistant"):
                    st.write(error_message)

        except Exception as e:
            st.error(f"An error occurred: {e}")

if __name__ == "__main__":
    main()

To run this program

1. Save the above three parts in a single app.py file

2. Create a requirements.txt file with the following content:

streamlit
requests
Pillow
python-dotenv

3. Setting Up and Running

Set Up a Virtual Environment (Optional):

python -m venv venv
source venv/bin/activate  # For Windows: `venv\Scripts\activate`

4. Install Dependencies

pip install -r requirements.txt

5. Run the app

streamlit run app.py

6.🎉 Try the Live App Live app in your browser.

Summary

Part 1: Sets up essential libraries, authentication, and utility functions. Part 2: Manages user input, file uploads, and chat session states. Part 3: Integrates with IBM’s Watson AI API and handles AI-driven responses.

Now you can build your app to combine visual data with conversational AI capabilities! 🚀

IBM watsonx.ai enables you to import machine learning models trained outside of its environment. The models can be stored in the watsonx.ai repository (a Cloud Object Storage bucket) and optionally deployed for testing.

Ways to Import Models

1. Directly through the UI
2. By using a path to a file
3. By using a path to a directory

Steps to Import a Model Using the UI

1. Navigate to the Assets tab in your watsonx.ai space.
2. Click Import assets.

1. Select Local file, then Model.

1. Choose the model file and click Import.
   • The system will automatically select a matching model type based on the version string in the file.

Supported Frameworks and Import Options

Note: Models in PMML format can be imported directly by uploading the .xml file.

Creating Batch Deployments

Batch deployments process input data from files or data connections and write the output to a specified destination. Unlike online deployments, batch deployments are designed for asynchronous processing.

Steps to Create a Batch Deployment from UI

1. Organize resources in a deployment space, adding deployable assets and data files.
2. Deploy the asset (e.g., machine learning model) with Batch as the deployment type.

1. Configure the batch deployment job by specifying the above in new job:

• Input data location
• Output data destination

• Scheduling details (if needed)

1. Click Create. The status will change to Deployed upon successful creation.

1. Run the job, which processes the input data and writes the output to the specified location.

Supported Asset Types for Batch Deployment

• Models: AutoAI, Scikit-learn, TensorFlow, XGBoost, Spark MLlib, PyTorch-ONNX, PMML, SPSS Modeler
• Scripts: Python scripts
• Functions: Python functions, Decision Optimization models

Testing Batch Deployments

1. Create a batch job from the deployment space.
2. Define the job, including input data and run schedule.
3. Run the job manually or as per the schedule.
4. View or download the output from the Assets page.

Conclusion

IBM watsonx.ai provides powerful tools for managing AI workflows, from importing models to executing batch deployments. Use this guide to streamline your AI model deployment and processing tasks.

Happy Deploying! 🎉

The ability to extract meaningful information from multiple document types (like PDFs, DOCX, CSV, JSON, and more) has become essential for businesses and researchers. This blog explains how to build a Streamlit app that integrates Watsonx.ai, LangChain, and retrieval-augmented generation (RAG) to make querying documents seamless and efficient.

Live App

Link to live app

What is RAG?

Retrieval-Augmented Generation (RAG) is a technique that combines document retrieval with large language models (LLMs) to generate accurate and context-based responses. It retrieves relevant information from your documents before generating answers, making it highly effective for specialized queries.

What is Watsonx.ai?

IBM Watsonx.ai is IBM’s next-generation platform for foundation models and generative AI. It offers pre-trained language models that can be fine-tuned for tasks like document querying, answering questions, and more. In this project, Watsonx.ai acts as the backbone for generating context-aware answers.

What is LangChain?

LangChain is a framework for developing applications powered by LLMs. It simplifies tasks like document retrieval, question answering, and conversation handling by connecting LLMs with tools like embeddings and databases.

What is Streamlit?

Streamlit is a Python-based framework for building data-driven web apps quickly. It provides an intuitive interface for users to interact with your application, making it an ideal choice for creating this multi-document retrieval tool.

Features

• File Support: Supports multiple file formats such as PDFs, Word documents, PowerPoint presentations, CSV, JSON, YAML, HTML, and plain text.
• Watsonx LLM Integration: Utilize IBM Watsonx’s LLM models for querying and generating answers.
• Embeddings: Uses HuggingFace embeddings for document indexing.
• RAG (Retrieval Augmented Generation): Combines document-based retrieval with LLMs for accurate responses.
• Streamlit Interface: Provides an intuitive user experience.

Installation

Follow these steps to clone and run the project locally:

Prerequisites

1. Python 3.8+ installed on your system.
2. Install pip (Python package manager).
3. An IBM Watsonx API key and Project ID.
4. Install Git if not already installed.

Clone the Repository

git clone https://github.com/Abd-al-RahmanH/Multi-Doc-Retrieval-Watsonx.git
cd Multi-Doc-Retrieval-Watsonx

Install Dependencies

1. Create a virtual environment (optional but recommended):

    python -m venv env
    source env/bin/activate  # On Windows: .\env\Scripts\activate
   

2. Install required Python packages:

    pip install -r requirements.txt
   

Set Environment Variables

Create a .env file in the project directory with the following keys:

WATSONX_API_KEY=<your_watsonx_api_key>
WATSONX_PROJECT_ID=<your_watsonx_project_id>

Run the App

1. Start the Streamlit app by running:

    streamlit run app.py
   

2. Open the URL displayed in your terminal (usually http://localhost:8501) to access the app.

How to Use

1. Upload Documents: Drag and drop supported files (e.g., PDFs, DOCX, JSON) in the app sidebar.
2. Select Model and Parameters: Choose a Watsonx model and configure settings like output tokens and decoding methods.
3. Ask Questions: Enter queries in the chat input to retrieve answers based on the uploaded document.

Project Structure

Multi-Doc-Retrieval-Watsonx/
├── app.py               # Main application file
├── requirements.txt     # Python dependencies
├── README.md            # Project documentation
└── .env                 # Environment variables (not included in repo, create manually)

Dependencies

• Streamlit: For building the user interface.
• LangChain: For document retrieval and RAG implementation.
• HuggingFace Transformers: For embedding and vector representation.
• Watsonx Foundation Models: For querying and text generation.
• Various Python Libraries: For file handling, including pandas, python-docx, python-pptx, and more.

Contributing

We welcome contributions! If you’d like to improve this project:

1. Fork the repository.
2. Create a feature branch: git checkout -b feature-name.
3. Commit your changes: git commit -m 'Add a new feature'.
4. Push to the branch: git push origin feature-name.
5. Open a Pull Request.

More Blogs and Interesting Projects

For more blogs and interesting projects, visit my personal website: https://abdulrahmanh.com

License

This project is licensed under the MIT License. See the LICENSE file for details.

For guidance on creating an EC2 instance, check out my previous blog: How to Create an AWS EC2 Instance.Make sure the instance type bigger eg.t3.xlarge and allow All traffic

Prerequisites

Ensure that you have:

• An IBM Entitlement Key for Watsonx Data.
• An Ubuntu EC2 instance in AWS.

Step 1: Set Up Entitlement Key

1. Log in to your IBM container library.
2. Go to Add New key and create a new API key for entitlement.
3. Store the API key securely, as you’ll need it for the Watsonx Data installation.

Step 2: Install Docker

Watsonx Data requires Docker to manage its containers. Install Docker as follows:

# Update package information
sudo apt update

# Install Docker
sudo apt install -y docker.io

# Start Docker service
sudo systemctl start docker
sudo systemctl enable docker

Verify Docker installation:

docker --version

Step 3: Set Up Installation Directory and Environment Variables

Switch to root user, create an installation directory, and set the necessary environment variables.

sudo su -
mkdir watsonxdata
cd watsonxdata

# Set environment variables
export LH_ROOT_DIR=<NEW DIRECTORY-watsonxdata>
export LH_RELEASE_TAG=latest
export IBM_LH_TOOLBOX=cp.icr.io/cpopen/watsonx-data/ibm-lakehouse-toolbox:$LH_RELEASE_TAG
export LH_REGISTRY=cp.icr.io/cp/watsonx-data
export PROD_USER=cp
export IBM_ENTITLEMENT_KEY=<YOUR_IBM_ENTITLEMENT_KEY>
export IBM_ICR_IO=cp.icr.io

Replace <YOUR_IBM_ENTITLEMENT_KEY> with the key obtained in Step 1.

For Docker, set DOCKER_EXE as follows:

export DOCKER_EXE=docker

Step 4: Download and Extract Watsonx Data Developer Package

1. Pull the Watsonx Data developer package and copy it to the host system:

   $DOCKER_EXE pull $IBM_LH_TOOLBOX
   id=$($DOCKER_EXE create $IBM_LH_TOOLBOX)
   $DOCKER_EXE cp $id:/opt - > /tmp/pkg.tar
   $DOCKER_EXE rm $id
   

2. Extract the package and verify the checksum:

   tar -xf /tmp/pkg.tar -C /tmp
   cat /tmp/opt/bom.txt
   cksum /tmp/opt/*/*
   tar -xf /tmp/opt/dev/ibm-lh-dev-*.tgz -C $LH_ROOT_DIR
   

Step 5: Authenticate with IBM Registry

Log in to the IBM registry to authenticate and pull additional resources:

$DOCKER_EXE login ${IBM_ICR_IO} --username=${PROD_USER} --password=${IBM_ENTITLEMENT_KEY}

Step 6: Run Setup Script

Run the setup script to initialize the Watsonx Data Developer environment. You can set a custom password with the --password option; otherwise, the default password is password.

$LH_ROOT_DIR/ibm-lh-dev/bin/setup --license_acceptance=y --runtime=$DOCKER_EXE

Step 7: Start Watsonx Data Containers

Start the Watsonx Data containers using the following command:

$LH_ROOT_DIR/ibm-lh-dev/bin/start

Step 8: Access Watsonx Data Console

1. Open the Watsonx Data console by visiting https://<YOUR_EC2_PUBLIC_IP>:9443 (or the port specified during setup).
2. Log in with the username ibmlhadmin and the password you set during setup (default is password).

Managing Watsonx Data

Check Container Status

To view the status of all containers:

$LH_ROOT_DIR/ibm-lh-dev/bin/status --all

Stop All Containers

To stop all containers:

$LH_ROOT_DIR/ibm-lh-dev/bin/stop

Start/Stop a Specific Container

To manage individual containers, use stop_service and start_service commands. Replace <container_name> with the name from the docker ps output:

$LH_ROOT_DIR/ibm-lh-dev/bin/stop_service <container_name>
$LH_ROOT_DIR/ibm-lh-dev/bin/start_service <container_name>

Step 9: Log In to Watsonx Data

Once Watsonx Data is up and running, access the login page via your browser at https://<YOUR_EC2_PUBLIC_IP>:9443.

Login Page: Enter your username and password to access the Watsonx Data console.

After that you will see the Dashboard

Step 10: Infrastructure Manager

After logging in, navigate to the Infrastructure Manager to monitor and manage system resources and services.

Infrastructure Manager: View and control Watsonx Data’s underlying infrastructure and resource allocations.

Step 11: Explore the Query Workspace

Use the Query Workspace to write and run SQL queries directly within Watsonx Data.

Query Workspace: Execute SQL queries and analyze data with Watsonx Data’s SQL editor.

Step 12: Access Query History

The Query History section lets you review past queries, making it easy to track, repeat, or debug previous SQL commands.

Query History: Review and manage past queries for efficient workflow management.

Congratulations! You have successfully installed and configured IBM Watsonx Data 2.0 Developer Edition on your Ubuntu EC2 instance.

Resources:

• IBM Watsonx Data Documentation
• Docker Documentation

For more insights, check out my Blog Section.