Name	Name	Last commit message	Last commit date
parent directory ..
qemu	qemu
scripts	scripts
README.md	README.md

The original work can be found at qemu-hdl-cosim

Overview of qemu-hdl-cosim
Install Qemu and Create a VM image
Run Co-Simulation

Overview

This folder contains the necessary patch to QEMU and script to launch the VM with a FPGA cosim card attached.

Prerequisites

This release has been tested with the following tools:

Ubuntu 18.04.3
Vivado 2018.3
QEMU 2.10 rc3

Environment variables to set

COSIM_REPO_HOME -> Root of the source release
COSIM_PORT -> Any 4-digit number. If you want to run multiple instances on 
             the same machine, each one needs to have a different number.

Install Qemu and Create a VM image

Compile QEMU

Install Dependencies:

   sudo apt update && sudo apt upgrade
   sudo apt-get install build-essential autoconf libtool python vim 
   sudo apt-get install libzmq3-dev libczmq-dev libncurses5-dev libncursesw5-dev libsdl2-dev

Download QEMU 2.10 rc3

   cd $COSIM_REPO_HOME/qemu
   wget http://download.qemu-project.org/qemu-2.10.0-rc3.tar.xz
   tar -xJf qemu-2.10.0-rc3.tar.xz

Apply the patches

Apply one patch for the co-simulation and another patch for memfd.c.

   patch -s -p0 < qemu-cosim.patch
   patch -s -p0 < qemu-memfd.patch

Configure and build

   cd qemu-2.10.0-rc3
   mkdir build
   cd build
   ../configure --target-list=x86_64-softmmu --disable-vnc --enable-sdl --enable-curses

   sed -i 's/LIBS+=/LIBS+=-lzmq -lczmq /g' config-host.mak

   make -j32

Copy the launch script

   cp ../../../scripts/launch_fpga.sh .
   cd ../../

Create a QEMU image

Create a QEMU image file called cosim.qcow2 in $COSIM_REPO_HOME/qemu and install Ubuntu 16.04.3.

   qemu-2.10.0-rc3/build/qemu-img create -f qcow2 cosim.qcow2 16G
   sudo qemu-2.10.0-rc3/build/x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom /path/to/ubuntu-16.04.6-server-amd64.iso -smp cpus=2 -accel kvm -m 4096 -hda cosim.qcow2
   (name: user; passwd: user)

Launch QEMU in one terminal

   cd $COSIM_REPO_HOME/qemu/qemu-2.10.0-rc3/build
   ./launch_fpga.sh
   (sudo -E x86_64-softmmu/qemu-system-x86_64 -m 4G -enable-kvm -cpu host -smp cores=1 -drive file=../../cosim.qcow2,cache=writethrough -device accelerator-pcie -redir tcp:2200::22 -display none)

   ssh -p 2200 user@localhost

In the VM, Install necessary tools for compiling userspace program and kernel module

   sudo apt-get update
   sudo apt-get upgrade -y
   sudo apt-get install build-essential

Copy driver and application to the image

Copy opae-intel-fpga-driver to the image.

   cd $COSIM_REPO_HOME
   scp -P 2200 -r ../../OPAE/driver/opae-intel-fpga-driver-1.3.0-2_pcie/ user@localhost:/home/user/.

Copy the Docker scripts to the Guest Machine, to run Docker VM on the QEMU VM(Optional)

In the host, Copy the Docker script to the image.

   cd $COSIM_REPO_HOME
   scp -P 2200 -r ../../Docker/ubuntu1x04 user@localhost:/home/user/.

In the VM, Follow the README.md to create the Docker VM and install drivers and SDK in the Docker VM, without polluting the QEMU VM.

Shutdown and Backup the image(Optional)

In the VM, Shutdown the VM

   sudo poweroff

In the host, Backup the installed image

   cd $COSIM_REPO_HOME/qemu
   zip cosim.qcow2.zip cosim.qcow2

Upgrade kernel(Ubuntu 18.04 only, reboot needed)

In the VM, upgrade the Ubuntu 18.04 kernel to version 5.3

   sudo apt-get install --install-recommends linux-generic-hwe-18.04
   sudo poweroff

Caution: In kernel 4.17 and above, the DFL driver will be loaded and will access the simulated HW. So after the kernel is upgraded, the Vivado XSim must be launched before booting the image. (Follow the procedure below.)

Run co-simulation

Run Vivado XSim in Host Machine

In the host, Launch Vivado XSim Simulation in the 1st terminal

   cd $COSIM_REPO_HOME/../../../hw/prj/qemu_hdl_cosim/
   make build-cosim

The waveform window will show AXI transactions when the application is launched in the VM.

Run application in Guest Machine

In the host, Launch QEMU with accelerator in the 2nd terminal

   cd $COSIM_REPO_HOME/qemu/qemu-2.10.0-rc3/build
   ./launch_fpga.sh
   (sudo -E x86_64-softmmu/qemu-system-x86_64 -m 4G -enable-kvm -cpu host -smp cores=1 -drive file=../../cosim.qcow2,cache=writethrough -device accelerator-pcie -redir tcp:2200::22 -display none)

In the host, Log in to the VM in the 3rd terminal

   ssh -p 2200 user@localhost

In the VM, compile and load driver

   cd opae-intel-fpga-driver-1.3.0-2_pcie/
   ./loadRunModule.sh

Printout

The printout will be like this:

   user@ubuntu1604-opae-vfpga:~/opae-intel-fpga-driver-1.3.0-2_pcie$ ./loadRunModule.sh 
   	 Compiling the driver 
   
   make -C /lib/modules/4.4.0-142-generic/build M=/home/user/opae-intel-fpga-driver-1.3.0-2_pcie clean
   make[1]: Entering directory '/usr/src/linux-headers-4.4.0-142-generic'
     CLEAN   /home/user/opae-intel-fpga-driver-1.3.0-2_pcie/.tmp_versions
     CLEAN   /home/user/opae-intel-fpga-driver-1.3.0-2_pcie/Module.symvers
   make[1]: Leaving directory '/usr/src/linux-headers-4.4.0-142-generic'
   make -C /lib/modules/4.4.0-142-generic/build M=/home/user/opae-intel-fpga-driver-1.3.0-2_pcie modules
   make[1]: Entering directory '/usr/src/linux-headers-4.4.0-142-generic'
     CC [M]  /home/user/opae-intel-fpga-driver-1.3.0-2_pcie/drivers/fpga/intel/uuid_mod.o
     CC [M]  /home/user/opae-intel-fpga-driver-1.3.0-2_pcie/drivers/fpga/intel/pcie.o
     CC [M]  /home/user/opae-intel-fpga-driver-1.3.0-2_pcie/drivers/fpga/intel/pcie_check.o
     CC [M]  /home/user/opae-intel-fpga-driver-1.3.0-2_pcie/drivers/fpga/intel/feature-dev.o
     LD [M]  /home/user/opae-intel-fpga-driver-1.3.0-2_pcie/intel-fpga-pci.o
     Building modules, stage 2.
     MODPOST 1 modules
     CC      /home/user/opae-intel-fpga-driver-1.3.0-2_pcie/intel-fpga-pci.mod.o
     LD [M]  /home/user/opae-intel-fpga-driver-1.3.0-2_pcie/intel-fpga-pci.ko
   make[1]: Leaving directory '/usr/src/linux-headers-4.4.0-142-generic'
   	 Before loading. 
   
   [sudo] password for user: 
   rmmod: ERROR: Module intel_fpga_pci is not currently loaded
   
   	 **** Loading the pcie module 
   
   [  225.511563] intel_fpga_pci: loading out-of-tree module taints kernel.
   [  225.511613] intel_fpga_pci: module verification failed: signature and/or required key missing - tainting kernel
   [  225.512957] LOG: call_stack: ccidrv_init
   [  225.512960] Intel(R) FPGA PCIe Driver: Version 0.14.0
   [  225.512962] LOG: call_stack: fpga_ids_init
   [  225.513694] LOG: call_stack: cci_pci_probe
   [  225.513766] intel-fpga-pci 0000:00:04.0: PCIE AER unavailable -5.
   [  225.514383] LOG: call_stack: create_init_drvdata
   [  225.514766] LOG: call_stack: alloc_fpga_id
   [  225.514769] LOG: call_stack: cci_pci_alloc_irq
   [  225.514770] LOG: call_stack: cci_pci_create_feature_devs
   [  225.514771] LOG: call_stack: build_info_alloc_and_init
   [  225.514772] LOG: call_stack: fpga_create_parent_dev
   [  225.515763] LOG: call_stack: parse_start
   [  225.515764] LOG: call_stack: parse_start_from
   [  225.515766] LOG: call_stack: cci_pci_ioremap_bar
   [  225.515788] LOG: call_stack: parse_feature_list
   [  225.515789] LOG: call_stack: parse_feature_list, for_loop
   [  225.515790] LOG: call_stack: parse_feature
   [  225.515791] LOG: readq: header.csr = readq(hdr);
   [  225.558231] LOG: call_stack: parse_feature_fiu
   [  225.558234] LOG: readq: header.csr = readq(hdr);
   [  225.595050] LOG: call_stack: parse_feature_port
   [  225.595061] LOG: call_stack: build_info_create_dev
   [  225.595062] LOG: call_stack: build_info_commit_dev
   [  225.595497] LOG: call_stack: alloc_fpga_id
   [  225.595501] LOG: call_stack: create_feature_instance
   [  225.624598] LOG: call_stack: build_info_add_sub_feature
   [  225.661200] LOG: call_stack: enable_port_uafu
   [  225.661203] LOG: readq: capability.csr = readq(&port_hdr->capability);
   [  225.661204] LOG: readq: control.csr = readq(&port_hdr->control);
   [  225.976359] LOG: readq: fiu_header.csr = readq(&fiu_hdr->csr);
   [  226.012213] LOG: readq: header.csr = readq(hdr);
   [  226.048678] LOG: call_stack: parse_ports_from_fme
   [  226.048680] LOG: call_stack: build_info_commit_dev
   [  226.050592] LOG: call_stack: feature_dev_id_type
   [  226.050594] LOG: call_stack: cci_pci_add_port_dev
   user@ubuntu1604-opae-vfpga:~/opae-intel-fpga-driver-1.3.0-2_pcie$

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

qemu_hdl_cosim

qemu_hdl_cosim

README.md

Table of Contents

Overview

Prerequisites

Environment variables to set

Install Qemu and Create a VM image

Compile QEMU

Create a QEMU image

Copy driver and application to the image

Copy the Docker scripts to the Guest Machine, to run Docker VM on the QEMU VM(Optional)

Shutdown and Backup the image(Optional)

Upgrade kernel(Ubuntu 18.04 only, reboot needed)

Run co-simulation

Run Vivado XSim in Host Machine

Run application in Guest Machine

Printout

Files

qemu_hdl_cosim

Directory actions

More options

Directory actions

More options

Latest commit

History

qemu_hdl_cosim

Folders and files

parent directory

README.md

Table of Contents

Overview

Prerequisites

Environment variables to set

Install Qemu and Create a VM image

Compile QEMU

Create a QEMU image

Copy driver and application to the image

Copy the Docker scripts to the Guest Machine, to run Docker VM on the QEMU VM(Optional)

Shutdown and Backup the image(Optional)

Upgrade kernel(Ubuntu 18.04 only, reboot needed)

Run co-simulation

Run Vivado XSim in Host Machine

Run application in Guest Machine

Printout