Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 1 | FU540-C000 RISC-V SoC |
| 2 | ===================== |
| 3 | The FU540-C000 is the world’s first 4+1 64-bit RISC‑V SoC from SiFive. |
| 4 | |
| 5 | The HiFive Unleashed development platform is based on FU540-C000 and capable |
| 6 | of running Linux. |
| 7 | |
| 8 | Mainline support |
| 9 | ================ |
| 10 | The support for following drivers are already enabled: |
| 11 | 1. SiFive UART Driver. |
| 12 | 2. SiFive PRCI Driver for clock. |
| 13 | 3. Cadence MACB ethernet driver for networking support. |
| 14 | |
| 15 | TODO: |
Anup Patel | 8911a22 | 2019-07-17 04:23:53 +0000 | [diff] [blame^] | 16 | 1. U-Boot expects the serial console device entry to be present under /chosen |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 17 | DT node. Example: |
| 18 | chosen { |
| 19 | stdout-path = "/soc/serial@10010000:115200"; |
| 20 | }; |
| 21 | |
| 22 | Without a serial console U-Boot will panic. |
| 23 | |
| 24 | Building |
| 25 | ======== |
| 26 | 1. Add the RISC-V toolchain to your PATH. |
| 27 | 2. Setup ARCH & cross compilation enviornment variable. |
| 28 | a. export ARCH=riscv |
| 29 | b. export CROSS_COMPILE=<riscv64 toolchain prefix> |
| 30 | 3. make sifive_fu540_defconfig |
| 31 | 4. make |
| 32 | |
| 33 | Flashing |
| 34 | ======== |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 35 | The current U-Boot port is supported in S-mode only and loaded directly |
| 36 | into DRAM. |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 37 | |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 38 | A prior stage (M-mode) firmware/bootloader (e.g OpenSBI) is required to |
| 39 | boot the u-boot.bin in S-mode and provide M-mode runtime services. |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 40 | |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 41 | Currently, the u-boot.bin is used as a payload of the OpenSBI FW_PAYLOAD |
| 42 | firmware. We need to compile OpenSBI with below command: |
| 43 | make PLATFORM=sifive/fu540 FW_PAYLOAD_PATH=<path to u-boot.bin> FW_PAYLOAD_FDT_PATH=<path to hifive-unleashed-a00.dtb from Linux> |
| 44 | (Note: Prefer hifive-unleashed-a00.dtb from Linux-5.3 or higher) |
| 45 | (Note: Linux-5.2 is also fine but it does not have ethernet DT node) |
| 46 | |
| 47 | More detailed description of steps required to build FW_PAYLOAD firmware |
| 48 | is beyond the scope of this document. Please refer OpenSBI documenation. |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 49 | (Note: OpenSBI git repo is at https://github.com/riscv/opensbi.git) |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 50 | |
| 51 | Once the prior stage firmware/bootloader binary is generated, it should be |
| 52 | copied to the first partition of the sdcard. |
| 53 | |
| 54 | sudo dd if=<prior_stage_firmware_binary> of=/dev/disk2s1 bs=1024 |
| 55 | |
| 56 | Booting |
| 57 | ======= |
| 58 | Once you plugin the sdcard and power up, you should see the U-Boot prompt. |
| 59 | |
| 60 | Sample boot log from HiFive Unleashed board |
| 61 | =========================================== |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 62 | U-Boot 2019.07-rc4-00013-g1837f893b0 (Jun 20 2019 - 11:08:48 +0530) |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 63 | |
| 64 | CPU: rv64imafdc |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 65 | Model: SiFive HiFive Unleashed A00 |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 66 | DRAM: 8 GiB |
| 67 | In: serial@10010000 |
| 68 | Out: serial@10010000 |
| 69 | Err: serial@10010000 |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 70 | Net: eth0: ethernet@10090000 |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 71 | Hit any key to stop autoboot: 0 |
| 72 | => version |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 73 | U-Boot 2019.07-rc4-00013-g1837f893b0 (Jun 20 2019 - 11:08:48 +0530) |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 74 | |
| 75 | riscv64-linux-gcc.br_real (Buildroot 2018.11-rc2-00003-ga0787e9) 8.2.0 |
| 76 | GNU ld (GNU Binutils) 2.31.1 |
| 77 | => |
| 78 | =============================================================================== |
| 79 | |
| 80 | Now you can configure your networking, tftp server and use tftp boot method to |
| 81 | load uImage. |
| 82 | |
| 83 | ========================================================================== |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 84 | => setenv ipaddr 10.206.5.241 |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 85 | => setenv netmask 255.255.252.0 |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 86 | => setenv serverip 10.206.4.143 |
| 87 | => setenv gateway 10.206.4.1 |
| 88 | => tftpboot ${kernel_addr_r} /sifive/fu540/uImage |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 89 | ethernet@10090000: PHY present at 0 |
| 90 | ethernet@10090000: Starting autonegotiation... |
| 91 | ethernet@10090000: Autonegotiation complete |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 92 | ethernet@10090000: link up, 1000Mbps full-duplex (lpa: 0x7c00) |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 93 | Using ethernet@10090000 device |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 94 | TFTP from server 10.206.4.143; our IP address is 10.206.5.241 |
| 95 | Filename '/sifive/fu540/uImage'. |
| 96 | Load address: 0x80600000 |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 97 | Loading: ################################################################# |
| 98 | ################################################################# |
| 99 | ################################################################# |
| 100 | ################################################################# |
| 101 | ################################################################# |
| 102 | ################################################################# |
| 103 | ################################################################# |
| 104 | ################################################################# |
| 105 | ################################################################# |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 106 | ######################################## |
| 107 | 1.5 MiB/s |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 108 | done |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 109 | Bytes transferred = 9162364 (8bce7c hex) |
| 110 | => tftpboot ${ramdisk_addr_r} /sifive/fu540/uRamdisk |
| 111 | ethernet@10090000: PHY present at 0 |
| 112 | ethernet@10090000: Starting autonegotiation... |
| 113 | ethernet@10090000: Autonegotiation complete |
| 114 | ethernet@10090000: link up, 1000Mbps full-duplex (lpa: 0x7c00) |
| 115 | Using ethernet@10090000 device |
| 116 | TFTP from server 10.206.4.143; our IP address is 10.206.5.241 |
| 117 | Filename '/sifive/fu540/uRamdisk'. |
| 118 | Load address: 0x82500000 |
| 119 | Loading: ################################################################# |
| 120 | ################################################################# |
| 121 | ################################## |
| 122 | 448.2 KiB/s |
| 123 | done |
| 124 | Bytes transferred = 2398272 (249840 hex) |
| 125 | => setenv bootargs "root=/dev/ram rw console=ttySIF0 earlycon=sbi" |
| 126 | => bootm ${kernel_addr_r} ${ramdisk_addr_r} ${fdtcontroladdr} |
| 127 | ## Booting kernel from Legacy Image at 80600000 ... |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 128 | Image Name: Linux |
| 129 | Image Type: RISC-V Linux Kernel Image (uncompressed) |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 130 | Data Size: 9162300 Bytes = 8.7 MiB |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 131 | Load Address: 80200000 |
| 132 | Entry Point: 80200000 |
| 133 | Verifying Checksum ... OK |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 134 | ## Loading init Ramdisk from Legacy Image at 82500000 ... |
| 135 | Image Name: Linux RootFS |
| 136 | Image Type: RISC-V Linux RAMDisk Image (uncompressed) |
| 137 | Data Size: 2398208 Bytes = 2.3 MiB |
| 138 | Load Address: 00000000 |
| 139 | Entry Point: 00000000 |
| 140 | Verifying Checksum ... OK |
| 141 | ## Flattened Device Tree blob at ff795730 |
| 142 | Booting using the fdt blob at 0xff795730 |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 143 | Loading Kernel Image ... OK |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 144 | Using Device Tree in place at 00000000ff795730, end 00000000ff799dac |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 145 | |
| 146 | Starting kernel ... |
| 147 | |
| 148 | [ 0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000 |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 149 | [ 0.000000] Linux version 5.2.0-rc1-00003-gb9543e66e700 (anup@anup-lab-machine) (gcc version 8.2.0 (Buildroot 2018.11-rc2-00003-ga0787e9)) #1 SMP Thu Jun 20 11:41:26 IST 2019 |
| 150 | [ 0.000000] earlycon: sbi0 at I/O port 0x0 (options '') |
| 151 | [ 0.000000] printk: bootconsole [sbi0] enabled |
| 152 | [ 0.000000] Initial ramdisk at: 0x(____ptrval____) (2398208 bytes) |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 153 | [ 0.000000] Zone ranges: |
| 154 | [ 0.000000] DMA32 [mem 0x0000000080200000-0x00000000ffffffff] |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 155 | [ 0.000000] Normal [mem 0x0000000100000000-0x000000027fffffff] |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 156 | [ 0.000000] Movable zone start for each node |
| 157 | [ 0.000000] Early memory node ranges |
| 158 | [ 0.000000] node 0: [mem 0x0000000080200000-0x000000027fffffff] |
| 159 | [ 0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x000000027fffffff] |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 160 | [ 0.000000] software IO TLB: mapped [mem 0xfb795000-0xff795000] (64MB) |
| 161 | [ 0.000000] CPU with hartid=0 is not available |
| 162 | [ 0.000000] CPU with hartid=0 is not available |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 163 | [ 0.000000] elf_hwcap is 0x112d |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 164 | [ 0.000000] percpu: Embedded 17 pages/cpu s29592 r8192 d31848 u69632 |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 165 | [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 2067975 |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 166 | [ 0.000000] Kernel command line: root=/dev/ram rw console=ttySIF0 earlycon=sbi |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 167 | [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes) |
| 168 | [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes) |
| 169 | [ 0.000000] Sorting __ex_table... |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 170 | [ 0.000000] Memory: 8182056K/8386560K available (5753K kernel code, 357K rwdata, 1804K rodata, 204K init, 808K bss, 204504K reserved, 0K cma-reserved) |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 171 | [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 |
| 172 | [ 0.000000] rcu: Hierarchical RCU implementation. |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 173 | [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4. |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 174 | [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 175 | [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 |
| 176 | [ 0.000000] NR_IRQS: 0, nr_irqs: 0, preallocated irqs: 0 |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 177 | [ 0.000000] plic: mapped 53 interrupts with 4 handlers for 9 contexts. |
| 178 | [ 0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [2] |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 179 | [ 0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x1d854df40, max_idle_ns: 3526361616960 ns |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 180 | [ 0.000007] sched_clock: 64 bits at 1000kHz, resolution 1000ns, wraps every 2199023255500ns |
| 181 | [ 0.008553] Console: colour dummy device 80x25 |
| 182 | [ 0.012990] Calibrating delay loop (skipped), value calculated using timer frequency.. 2.00 BogoMIPS (lpj=4000) |
| 183 | [ 0.023103] pid_max: default: 32768 minimum: 301 |
| 184 | [ 0.028269] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes) |
| 185 | [ 0.035068] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes) |
| 186 | [ 0.042770] *** VALIDATE proc *** |
| 187 | [ 0.045610] *** VALIDATE cgroup1 *** |
| 188 | [ 0.049157] *** VALIDATE cgroup2 *** |
| 189 | [ 0.053743] rcu: Hierarchical SRCU implementation. |
| 190 | [ 0.058297] smp: Bringing up secondary CPUs ... |
| 191 | [ 0.064134] smp: Brought up 1 node, 4 CPUs |
| 192 | [ 0.069114] devtmpfs: initialized |
| 193 | [ 0.073281] random: get_random_u32 called from bucket_table_alloc.isra.10+0x4e/0x160 with crng_init=0 |
| 194 | [ 0.082157] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns |
| 195 | [ 0.091634] futex hash table entries: 1024 (order: 4, 65536 bytes) |
| 196 | [ 0.098480] NET: Registered protocol family 16 |
| 197 | [ 0.114101] vgaarb: loaded |
| 198 | [ 0.116397] SCSI subsystem initialized |
| 199 | [ 0.120358] usbcore: registered new interface driver usbfs |
| 200 | [ 0.125541] usbcore: registered new interface driver hub |
| 201 | [ 0.130936] usbcore: registered new device driver usb |
| 202 | [ 0.136618] clocksource: Switched to clocksource riscv_clocksource |
| 203 | [ 0.148108] NET: Registered protocol family 2 |
| 204 | [ 0.152358] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes) |
| 205 | [ 0.159928] TCP established hash table entries: 65536 (order: 7, 524288 bytes) |
| 206 | [ 0.169027] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) |
| 207 | [ 0.178360] TCP: Hash tables configured (established 65536 bind 65536) |
| 208 | [ 0.184653] UDP hash table entries: 4096 (order: 5, 131072 bytes) |
| 209 | [ 0.190819] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes) |
| 210 | [ 0.197618] NET: Registered protocol family 1 |
| 211 | [ 0.201892] RPC: Registered named UNIX socket transport module. |
| 212 | [ 0.207395] RPC: Registered udp transport module. |
| 213 | [ 0.212159] RPC: Registered tcp transport module. |
| 214 | [ 0.216940] RPC: Registered tcp NFSv4.1 backchannel transport module. |
| 215 | [ 0.223445] PCI: CLS 0 bytes, default 64 |
| 216 | [ 0.227726] Unpacking initramfs... |
| 217 | [ 0.260556] Freeing initrd memory: 2336K |
| 218 | [ 0.264652] workingset: timestamp_bits=62 max_order=21 bucket_order=0 |
| 219 | [ 0.278452] NFS: Registering the id_resolver key type |
| 220 | [ 0.282841] Key type id_resolver registered |
| 221 | [ 0.287067] Key type id_legacy registered |
| 222 | [ 0.291155] nfs4filelayout_init: NFSv4 File Layout Driver Registering... |
| 223 | [ 0.298299] NET: Registered protocol family 38 |
| 224 | [ 0.302470] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 254) |
| 225 | [ 0.309906] io scheduler mq-deadline registered |
| 226 | [ 0.314501] io scheduler kyber registered |
| 227 | [ 0.354134] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled |
| 228 | [ 0.360725] 10010000.serial: ttySIF0 at MMIO 0x10010000 (irq = 4, base_baud = 0) is a SiFive UART v0 |
| 229 | [ 0.369191] printk: console [ttySIF0] enabled |
| 230 | [ 0.369191] printk: console [ttySIF0] enabled |
| 231 | [ 0.377938] printk: bootconsole [sbi0] disabled |
| 232 | [ 0.377938] printk: bootconsole [sbi0] disabled |
| 233 | [ 0.387298] 10011000.serial: ttySIF1 at MMIO 0x10011000 (irq = 1, base_baud = 0) is a SiFive UART v0 |
| 234 | [ 0.396411] [drm] radeon kernel modesetting enabled. |
| 235 | [ 0.409818] loop: module loaded |
| 236 | [ 0.412606] libphy: Fixed MDIO Bus: probed |
| 237 | [ 0.416870] macb 10090000.ethernet: Registered clk switch 'sifive-gemgxl-mgmt' |
| 238 | [ 0.423570] macb: GEM doesn't support hardware ptp. |
| 239 | [ 0.428469] libphy: MACB_mii_bus: probed |
| 240 | [ 1.053009] Microsemi VSC8541 SyncE 10090000.ethernet-ffffffff:00: attached PHY driver [Microsemi VSC8541 SyncE] (mii_bus:phy_addr=10090000.ethernet-ffffffff:00, irq=POLL) |
| 241 | [ 1.067548] macb 10090000.ethernet eth0: Cadence GEM rev 0x10070109 at 0x10090000 irq 7 (70:b3:d5:92:f2:f3) |
| 242 | [ 1.077330] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k |
| 243 | [ 1.083069] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. |
| 244 | [ 1.089061] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver |
| 245 | [ 1.095485] ehci-pci: EHCI PCI platform driver |
| 246 | [ 1.099947] ehci-platform: EHCI generic platform driver |
| 247 | [ 1.105196] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver |
| 248 | [ 1.111286] ohci-pci: OHCI PCI platform driver |
| 249 | [ 1.115742] ohci-platform: OHCI generic platform driver |
| 250 | [ 1.121142] usbcore: registered new interface driver uas |
| 251 | [ 1.126269] usbcore: registered new interface driver usb-storage |
| 252 | [ 1.132331] mousedev: PS/2 mouse device common for all mice |
| 253 | [ 1.137978] usbcore: registered new interface driver usbhid |
| 254 | [ 1.143325] usbhid: USB HID core driver |
| 255 | [ 1.148022] NET: Registered protocol family 10 |
| 256 | [ 1.152609] Segment Routing with IPv6 |
| 257 | [ 1.155571] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver |
| 258 | [ 1.161927] NET: Registered protocol family 17 |
| 259 | [ 1.165907] Key type dns_resolver registered |
| 260 | [ 1.171694] Freeing unused kernel memory: 204K |
| 261 | [ 1.175375] This architecture does not have kernel memory protection. |
| 262 | [ 1.181792] Run /init as init process |
| 263 | _ _ |
| 264 | | ||_| |
| 265 | | | _ ____ _ _ _ _ |
| 266 | | || | _ \| | | |\ \/ / |
| 267 | | || | | | | |_| |/ \ |
| 268 | |_||_|_| |_|\____|\_/\_/ |
Atish Patra | 8ad2104 | 2019-02-25 08:15:27 +0000 | [diff] [blame] | 269 | |
Anup Patel | 0e43631 | 2019-06-25 06:31:52 +0000 | [diff] [blame] | 270 | Busybox Rootfs |
| 271 | |
| 272 | Please press Enter to activate this console. |
| 273 | / # |