mirror of https://github.com/openXC7/prjxray.git
426 lines
16 KiB
Markdown
426 lines
16 KiB
Markdown
# Project X-Ray
|
||
|
||
[](http://prjxray.readthedocs.org/)
|
||
[](https://github.com/f4pga/prjxray/blob/master/LICENSE)
|
||
|
||
|
||
Documenting the Xilinx 7-series bit-stream format.
|
||
|
||
This repository contains both tools and scripts which allow you to document the
|
||
bit-stream format of Xilinx 7-series FPGAs.
|
||
|
||
More documentation can be found published on [prjxray ReadTheDocs site](https://prjxray.readthedocs.io/en/latest/) - this includes;
|
||
* [Highlevel Bitstream Architecture](https://prjxray.readthedocs.io/en/latest/architecture/overview.html)
|
||
* [Overview of DB Development Process](https://prjxray.readthedocs.io/en/latest/db_dev_process/index.html)
|
||
|
||
# Quickstart Guide
|
||
Instructions were originally written for Ubuntu 16.04. Please let us know if you have information on other distributions.
|
||
|
||
### Step 1: ###
|
||
Install Vivado 2017.2. If you did not install to /opt/Xilinx default, then set the environment variable
|
||
XRAY_VIVADO_SETTINGS to point to the settings64.sh file of the installed vivado version, ie
|
||
|
||
export XRAY_VIVADO_SETTINGS=/opt/Xilinx/Vivado/2017.2/settings64.sh
|
||
|
||
Do not source the settings64.sh in your shell, since this adds directories of
|
||
the Vivado installation at the beginning of your PATH and LD_LIBRARY_PATH
|
||
variables, which will likely interfere with or break non-Vivado applications in
|
||
that shell. The Vivado wrapper utils/vivado.sh makes sure that the environment
|
||
variables from XRAY_VIVADO_SETTINGS are automatically sourced in a separate
|
||
shell that is then only used to run Vivado to avoid these problems.
|
||
|
||
**Why 2017.2?** Currently the fuzzers only work on `2017.2`, see [Issue #14 on prjxray](https://github.com/f4pga/prjxray/issues/14).
|
||
|
||
**Is 2017.2 really required?** Yes, only `2017.2` works. Until Issue #14 is solved, **only** `2017.2` works and will be supported.
|
||
|
||
### Step 2: ###
|
||
Clone the ``prjxray`` repository and its submodules:
|
||
|
||
```bash
|
||
git clone git@github.com:f4pga/prjxray.git
|
||
cd prjxray
|
||
git submodule update --init --recursive
|
||
```
|
||
|
||
### Step 3: ###
|
||
Install CMake:
|
||
|
||
```bash
|
||
sudo apt-get install cmake # version 3.5.0 or later required,
|
||
# for Ubuntu Trusty pkg is called cmake3
|
||
```
|
||
|
||
### Step 4: ###
|
||
Build the C++ tools, in the prjxray root directory run:
|
||
|
||
```bash
|
||
make build
|
||
```
|
||
|
||
### Step 5: ###
|
||
Choose one of the following options:
|
||
|
||
(Option 1) - Install the Python environment locally
|
||
|
||
```bash
|
||
sudo apt-get install virtualenv python3 python3-pip python3-virtualenv python3-yaml
|
||
make env
|
||
```
|
||
|
||
(Option 2) - Install the Python environment globally
|
||
|
||
```bash
|
||
sudo apt-get install python3 python3-pip python3-yaml
|
||
sudo -H pip3 install -r requirements.txt
|
||
```
|
||
|
||
This step is known to fail with a compiler error while building the `pyjson5`
|
||
library when using Arch Linux and Fedora. If this occurs, `pyjson5` needs one
|
||
change to build correctly:
|
||
|
||
```bash
|
||
git clone https://github.com/Kijewski/pyjson5.git
|
||
cd pyjson5
|
||
sed -i 's/char \*PyUnicode/const char \*PyUnicode/' src/_imports.pyx
|
||
sudo make
|
||
```
|
||
|
||
This might give you an error about `sphinx_autodoc_typehints` but it should
|
||
correctly build and install pyjson5. After this, run either option 1 or 2 again.
|
||
|
||
### Step 6: ###
|
||
|
||
Prepare the database with static part information, which are needed by the
|
||
fuzzers, either for all device families
|
||
|
||
```bash
|
||
make db-prepare-parts
|
||
```
|
||
|
||
or only for a selected one
|
||
|
||
```bash
|
||
make db-prepare-artix7
|
||
```
|
||
|
||
### Step 7: ###
|
||
Always make sure to set the environment for the device you are working on before
|
||
running any other commands:
|
||
|
||
```bash
|
||
source settings/artix7.sh
|
||
```
|
||
|
||
### Step 8: ###
|
||
|
||
(Option 1, recommended) - Download a current stable version (you can use the
|
||
Python API with a pre-generated database)
|
||
|
||
```bash
|
||
./download-latest-db.sh
|
||
```
|
||
|
||
(Option 2) - (Re-)create the entire database (this will take a very long time!)
|
||
|
||
```bash
|
||
cd fuzzers
|
||
make -j$(nproc)
|
||
```
|
||
|
||
### Step 9: ###
|
||
Pick a fuzzer (or write your own), from the ``prjxray`` root dir, run:
|
||
|
||
```bash
|
||
cd fuzzers/010-clb-lutinit
|
||
make -j$(nproc) run
|
||
```
|
||
|
||
### Step 10: ###
|
||
Create HTML documentation, from the ``prjxray`` root dir, run:
|
||
|
||
```
|
||
cd htmlgen
|
||
python3 htmlgen.py
|
||
```
|
||
|
||
# C++ Development
|
||
|
||
Tests are not built by default. Setting the PRJXRAY\_BUILD\_TESTING option to
|
||
ON when running cmake will include them. From the ``prjxray`` root dir, run:
|
||
|
||
```bash
|
||
mkdir -p build
|
||
cd build
|
||
cmake -DPRJXRAY_BUILD_TESTING=ON ..
|
||
make
|
||
```
|
||
|
||
The default C++ build configuration is for releases (optimizations enabled, no
|
||
debug info). A build configuration for debugging (no optimizations, debug info)
|
||
can be chosen via the CMAKE\_BUILD\_TYPE option. From the ``prjxray`` root dir, run:
|
||
|
||
```bash
|
||
mkdir -p build
|
||
cd build
|
||
cmake -DCMAKE_BUILD_TYPE=ON ..
|
||
make
|
||
```
|
||
|
||
The options to build tests and use a debug build configuration are independent
|
||
to allow testing that optimizations do not cause bugs. The build configuration
|
||
and build tests options may be combined to allow all permutations.
|
||
|
||
# Process
|
||
|
||
The documentation is done through a "black box" process were Vivado is asked to
|
||
generate a large number of designs which then used to create bitstreams. The
|
||
resulting bit streams are then cross correlated to discover what different bits
|
||
do.
|
||
|
||
## Parts
|
||
|
||
### [Minitests](minitests)
|
||
|
||
There are also "minitests" which are designs which can be viewed by a human in
|
||
Vivado to better understand how to generate more useful designs.
|
||
|
||
### [Experiments](experiments)
|
||
|
||
Experiments are like "minitests" except are only useful for a short period of
|
||
time. Files are committed here to allow people to see how we are trying to
|
||
understand the bitstream.
|
||
|
||
When an experiment is finished with, it will be moved from this directory into
|
||
the latest "prjxray-experiments-archive-XXXX" repository.
|
||
|
||
### [Fuzzers](fuzzers)
|
||
|
||
Fuzzers are the scripts which generate the large number of bitstream.
|
||
|
||
They are called "fuzzers" because they follow an approach similar to the
|
||
[idea of software testing through fuzzing](https://en.wikipedia.org/wiki/Fuzzing).
|
||
|
||
### [Tools](tools) & [Libs](lib)
|
||
|
||
Tools & libs are useful tools (and libraries) for converting the resulting
|
||
bitstreams into various formats.
|
||
|
||
Binaries in the tools directory are considered more mature and stable then
|
||
those in the [utils](utils) directory and could be actively used in other
|
||
projects.
|
||
|
||
### [Utils](utils)
|
||
|
||
Utils are various tools which are still highly experimental. These tools should
|
||
only be used inside this repository.
|
||
|
||
### [Third Party](third_party)
|
||
|
||
Third party contains code not developed as part of Project X-Ray.
|
||
|
||
|
||
# Database
|
||
|
||
Running the all fuzzers in order will produce a database which documents the
|
||
bitstream format in the [database](database) directory.
|
||
|
||
As running all these fuzzers can take significant time,
|
||
[Tim 'mithro' Ansell <me@mith.ro>](https://github.com/mithro) has graciously
|
||
agreed to maintain a copy of the database in the
|
||
[prjxray-db](https://github.com/f4pga/prjxray-db) repository.
|
||
|
||
Please direct enquires to [Tim](mailto:me@mith.ro) if there are any issues with
|
||
it.
|
||
|
||
# Current Focus
|
||
|
||
Current the focus has been on the Artix-7 50T part. This structure is common
|
||
between all footprints of the 15T, 35T and 50T varieties.
|
||
|
||
We have also started experimenting with the Kintex-7 and Virtex-7 parts.
|
||
|
||
The aim is to eventually document all parts in the Xilinx 7-series FPGAs but we
|
||
can not do this alone, **we need your help**!
|
||
|
||
## Virtex-7 Port Status (`virtex7-support` branch)
|
||
|
||
This branch ports the openXC7 prjxray flow to **Virtex-7 `xc7vx485tffg1761-2`**
|
||
(VC707 board), modelled on the Kintex-7 sub-flow. The goal is a fully
|
||
open-source bit→DCP / fasm→bit round-trip on a Virtex-7 HP-only part.
|
||
|
||
### Achievements
|
||
|
||
- **End-to-end smoke test passes** on `xc7vx485tffg1761-2`: System-Verilog
|
||
→ Yosys → nextpnr-xilinx → FASM → `fasm2frames.py` → `xc7frames2bit`
|
||
→ `.bit`. First run required 3 patches and surfaced 2 nextpnr-xilinx bugs
|
||
(filed upstream).
|
||
- **Cross-family segbits/ppips transplant** from the openXC7
|
||
[`prjxray-db`](https://github.com/openXC7/prjxray-db) Kintex-7 tree into
|
||
`database/virtex7/`: ~39 381 segbits + ~12 273 ppips entries, plus targeted
|
||
key copies for `clk_hrow_*`, `clk_bufg_*`, `hclk_cmt*`, `io_int_interface_*`
|
||
(≈51 k entries total). Pre-transplant tree backed up at
|
||
`database/virtex7.before_transplant/`.
|
||
- **11 fuzzers patched** for HP-bank / virtex7-grid awareness:
|
||
- `037-iob18-pips` — left-side mirror sites (LIOI / LIOI_TBYTESRC /
|
||
LIOI_TBYTETERM), `top.py` NOT_INCLUDED_TILES for `*_SING`, generate-side
|
||
tile-type normalization, `ioi_pip_list.tcl` LIOI emission.
|
||
- `039-hclk-config` — virtex7 split (HCLK_IOI vs HCLK_IOI3),
|
||
`XRAY_IOSTANDARD` env var, IOB18M / IOB33M alternation in `top.py`.
|
||
- `047a-hclk-idelayctrl-pips` — accepts both HCLK_IOI and HCLK_IOI3.
|
||
- `034 / 034b / 041 / 043 / 044 / 045 / 046` — removed local
|
||
`write_pip_txtdata` overrides that shadowed the patched `utils.tcl`
|
||
bulk-fetch (~4× faster per specimen on xc7vx485t).
|
||
- **`utils/utils.tcl` bulk-fetch patch** for `write_pip_txtdata` —
|
||
per-net `foreach pip` → bulk `get_pips` + bulk `get_property IS_DIRECTIONAL`
|
||
+ cached `dst_wire_to_num_pips`. ~4× speed-up; on
|
||
`xc7vx485tffg1761-2` cuts `041-clk-hrow-pips`/`045-hclk-cmt-pips` specimens
|
||
from ~1.5 h to ~25 min each.
|
||
- **`utils/fasm2frames.py` HP-bank fix** — was hard-coded to `HCLK_IOI3_*`
|
||
tile prefix; now probes the grid for whichever of `HCLK_IOI_` / `HCLK_IOI3_`
|
||
exists. STEPDOWN bank-anchor check widened accordingly.
|
||
- **`utils/mergedb.sh` extended** with LIOI / LIOI_TBYTESRC / LIOI_TBYTETERM
|
||
/ LIOB18 / mask_liob18 cases.
|
||
- **RapidWright `json2dcp.jar` rebuilt** against modern RapidWright (2025.2.1)
|
||
and patched for virtex7 — at `~/rapidwright/build/rapidwright_json2dcp.jar`
|
||
(17 KB). Verified `rst_to_led_routed.json` → 1.38 MB DCP that loads back
|
||
cleanly on `xc7vx485tffg1761-2`. Patch list in
|
||
`~/rapidwright/build/README.md`.
|
||
|
||
### Goals
|
||
|
||
- Document `xc7vx485tffg1761-2` bitstream format with parity sufficient for a
|
||
Vivado-equivalent bitstream on IOB-only designs first, then SLICE and CMT.
|
||
- Achieve bit ↔ bit equivalence with Vivado on the `rst_to_led` pass-through
|
||
reference, then incrementally on counter, BUFR/IDELAY, and CMT designs.
|
||
- Provide a runnable bit→DCP path so open-flow bitstreams can be inspected /
|
||
diffed against Vivado checkpoints (Phase A via `json2dcp.jar` done; Phase B
|
||
via a virtex7-aware `fasm2bels` still in progress).
|
||
|
||
### Work in Progress
|
||
|
||
- **030-iob18 N=200 rerun** for finer IOSTANDARD/DRIVE coverage on LIOB18.
|
||
Running at one specimen per ~3 min, currently ≈187 of 200 specimens written;
|
||
will trigger `make pushdb` automatically on completion.
|
||
- **LIOB18 IOSTANDARD bit-encoding bug** — `rst_to_led.bit` produced by our
|
||
flow flickers and is unresponsive on hardware; Vivado-built reference works.
|
||
Targeted bit-diff localised the discrepancy to LIOI and LIOI_TBYTETERM
|
||
tiles sharing a frame range but with different word offsets — partial fix
|
||
applied (per-tile partitioning) reduced missing bits 28 → 21 and extras
|
||
14 → 12, but IOSTANDARD bits still wrong. Awaiting N=200 segdata to
|
||
re-derive.
|
||
- **037-iob18-pips left-side coverage** — residual pip-convergence iteration;
|
||
iter 1 was salvaged after a JVM crash on `spec_008` left 19 surviving
|
||
specimens that yielded 128 LIOI entries.
|
||
- **Phase B — virtex7 `fasm2bels` port** — parked. Connection database for
|
||
`xc7vx485tffg1761-2` builds successfully (14 min, 4.2 GB on disk, 56.8 M
|
||
wires). First failure is in `clb_models.py` on the nextpnr-emitted
|
||
PSEUDO_VCC packer cell (`assert False, {'NOCLKINV'}`). IOB18 / HP-bank BEL
|
||
name substitutions drafted in `models/iob_models.py` (uncommitted).
|
||
- **Open TODOs (cross-cutting)** — expand the virtex7 ROI; add a Vivado DRC
|
||
cross-check of nextpnr output; exclude `GTX_INT_INTERFACE` pips from the
|
||
virtex7 chipdb; add bits info for `CFG_CENTER` / `*_SING` tiles.
|
||
|
||
### Constraints
|
||
|
||
- Patches must **not** regress other families (artix7 / kintex7 / zynq7 /
|
||
spartan7). The HP-bank additions are gated on tile-type prefix so the
|
||
HR-bank paths stay byte-identical.
|
||
- Build artefacts under `fuzzers/*/build/` are **retained** for debugging
|
||
— `make clean` is not invoked between fuzzer iterations.
|
||
|
||
## Adding a new part to an existing family
|
||
|
||
We have written a [detailed guide](https://f4pga.readthedocs.io/projects/prjxray/en/latest/db_dev_process/newpart.html) that walks through the process of adding a new part to an existing family.
|
||
|
||
## TODO List
|
||
|
||
- [ ] Write a TODO list
|
||
|
||
|
||
# Contributing
|
||
|
||
There are a couple of guidelines when contributing to Project X-Ray which are
|
||
listed here.
|
||
|
||
### Sending
|
||
|
||
All contributions should be sent as
|
||
[GitHub Pull requests](https://help.github.com/articles/creating-a-pull-request-from-a-fork/).
|
||
|
||
### License
|
||
|
||
All software (code, associated documentation, support files, etc) in the
|
||
Project X-Ray repository are licensed under the very permissive
|
||
[ISC Licence](https://opensource.org/licenses/ISC). A copy can be found in the [`LICENSE`](LICENSE) file.
|
||
|
||
All new contributions must also be released under this license.
|
||
|
||
### Code of Conduct
|
||
|
||
By contributing you agree to the [code of conduct](CODE_OF_CONDUCT.md). We
|
||
follow the open source best practice of using the [Contributor
|
||
Covenant](https://www.contributor-covenant.org/) for our Code of Conduct.
|
||
|
||
### Sign your work
|
||
|
||
To improve tracking of who did what, we follow the Linux Kernel's
|
||
["sign your work" system](https://github.com/wking/signed-off-by).
|
||
This is also called a
|
||
["DCO" or "Developer's Certificate of Origin"](https://developercertificate.org/).
|
||
|
||
**All** commits are required to include this sign off and we use the
|
||
[Probot DCO App](https://github.com/probot/dco) to check pull requests for
|
||
this.
|
||
|
||
The sign-off is a simple line at the end of the explanation for the
|
||
patch, which certifies that you wrote it or otherwise have the right to
|
||
pass it on as a open-source patch. The rules are pretty simple: if you
|
||
can certify the below:
|
||
|
||
Developer's Certificate of Origin 1.1
|
||
|
||
By making a contribution to this project, I certify that:
|
||
|
||
(a) The contribution was created in whole or in part by me and I
|
||
have the right to submit it under the open source license
|
||
indicated in the file; or
|
||
|
||
(b) The contribution is based upon previous work that, to the best
|
||
of my knowledge, is covered under an appropriate open source
|
||
license and I have the right under that license to submit that
|
||
work with modifications, whether created in whole or in part
|
||
by me, under the same open source license (unless I am
|
||
permitted to submit under a different license), as indicated
|
||
in the file; or
|
||
|
||
(c) The contribution was provided directly to me by some other
|
||
person who certified (a), (b) or (c) and I have not modified
|
||
it.
|
||
|
||
(d) I understand and agree that this project and the contribution
|
||
are public and that a record of the contribution (including all
|
||
personal information I submit with it, including my sign-off) is
|
||
maintained indefinitely and may be redistributed consistent with
|
||
this project or the open source license(s) involved.
|
||
|
||
then you just add a line saying
|
||
|
||
Signed-off-by: Random J Developer <random@developer.example.org>
|
||
|
||
using your real name (sorry, no pseudonyms or anonymous contributions.)
|
||
|
||
You can add the signoff as part of your commit statement. For example:
|
||
|
||
git commit --signoff -a -m "Fixed some errors."
|
||
|
||
*Hint:* If you've forgotten to add a signoff to one or more commits, you can use the
|
||
following command to add signoffs to all commits between you and the upstream
|
||
master:
|
||
|
||
git rebase --signoff upstream/master
|
||
|
||
### Contributing to the docs
|
||
|
||
In addition to the above contribution guidelines, see the guide to
|
||
[updating the Project X-Ray docs](UPDATING-THE-DOCS.md).
|