I'm putting together a Dom0 and three DomU (all Gentoo) with kernel 3.5.7 and
Xen 4.1.1. Each Dom has it's own md (md0 for Dom0, md1 for Dom1 etc). Dom0
works fine so far, however, I'm stuck trying to create DomUs. It appears the
xvda1 device on DomU is not created or accessible:
Parsing config file dom1
domainbuilder: detail: xc_dom_allocate: cmdline="root=/dev/xvda1 console=hvc0
root=/dev/xvda1 ro 3", features="(null)"
domainbuilder: detail: xc_dom_kernel_mem: called
domainbuilder: detail: xc_dom_boot_xen_init: ver 4.1, caps xen-3.0-x86_64
xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
domainbuilder: detail: xc_dom_parse_image: called
domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ...
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ...
domainbuilder: detail: xc_dom_malloc : 10530 kB
domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x2f7a4f -> 0xa48888
domainbuilder: detail: loader probe OK
xc: detail: elf_parse_binary: phdr: paddr=0x1000000 memsz=0x558000
xc: detail: elf_parse_binary: phdr: paddr=0x1558000 memsz=0x690e8
xc: detail: elf_parse_binary: phdr: paddr=0x15c2000 memsz=0x127c0
xc: detail: elf_parse_binary: phdr: paddr=0x15d5000 memsz=0x533000
xc: detail: elf_parse_binary: memory: 0x1000000 -> 0x1b08000
xc: detail: elf_xen_parse_note: GUEST_OS = "linux"
xc: detail: elf_xen_parse_note: GUEST_VERSION = "2.6"
xc: detail: elf_xen_parse_note: XEN_VERSION = "xen-3.0"
xc: detail: elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
xc: detail: elf_xen_parse_note: ENTRY = 0xffffffff815d5210
xc: detail: elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff81001000
xc: detail: elf_xen_parse_note: FEATURES =
"!writable_page_tables|pae_pgdir_above_4gb"
xc: detail: elf_xen_parse_note: PAE_MODE = "yes"
xc: detail: elf_xen_parse_note: LOADER = "generic"
xc: detail: elf_xen_parse_note: unknown xen elf note (0xd)
xc: detail: elf_xen_parse_note: SUSPEND_CANCEL = 0x1
xc: detail: elf_xen_parse_note: HV_START_LOW = 0xffff800000000000
xc: detail: elf_xen_parse_note: PADDR_OFFSET = 0x0
xc: detail: elf_xen_addr_calc_check: addresses:
xc: detail: virt_base = 0xffffffff80000000
xc: detail: elf_paddr_offset = 0x0
xc: detail: virt_offset = 0xffffffff80000000
xc: detail: virt_kstart = 0xffffffff81000000
xc: detail: virt_kend = 0xffffffff81b08000
xc: detail: virt_entry = 0xffffffff815d5210
xc: detail: p2m_base = 0xffffffffffffffff
domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_64:
0xffffffff81000000 -> 0xffffffff81b08000
domainbuilder: detail: xc_dom_mem_init: mem 5000 MB, pages 0x138800 pages, 4k each
domainbuilder: detail: xc_dom_mem_init: 0x138800 pages
domainbuilder: detail: xc_dom_boot_mem_init: called
domainbuilder: detail: x86_compat: guest xen-3.0-x86_64, address size 64
domainbuilder: detail: xc_dom_malloc : 10000 kB
domainbuilder: detail: xc_dom_build_image: called
domainbuilder: detail: xc_dom_alloc_segment: kernel : 0xffffffff81000000
-> 0xffffffff81b08000 (pfn 0x1000 + 0xb08 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x1000+0xb08 at
0x7fdec9b85000
xc: detail: elf_load_binary: phdr 0 at 0x0x7fdec9b85000 -> 0x0x7fdeca0dd000
xc: detail: elf_load_binary: phdr 1 at 0x0x7fdeca0dd000 -> 0x0x7fdeca1460e8
xc: detail: elf_load_binary: phdr 2 at 0x0x7fdeca147000 -> 0x0x7fdeca1597c0
xc: detail: elf_load_binary: phdr 3 at 0x0x7fdeca15a000 -> 0x0x7fdeca1cd000
domainbuilder: detail: xc_dom_alloc_segment: phys2mach : 0xffffffff81b08000
-> 0xffffffff824cc000 (pfn 0x1b08 + 0x9c4 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x1b08+0x9c4 at
0x7fdec91c1000
domainbuilder: detail: xc_dom_alloc_page : start info : 0xffffffff824cc000
(pfn 0x24cc)
domainbuilder: detail: xc_dom_alloc_page : xenstore : 0xffffffff824cd000
(pfn 0x24cd)
domainbuilder: detail: xc_dom_alloc_page : console : 0xffffffff824ce000
(pfn 0x24ce)
domainbuilder: detail: nr_page_tables: 0x0000ffffffffffff/48: 0xffff000000000000
-> 0xffffffffffffffff, 1 table(s)
domainbuilder: detail: nr_page_tables: 0x0000007fffffffff/39: 0xffffff8000000000
-> 0xffffffffffffffff, 1 table(s)
domainbuilder: detail: nr_page_tables: 0x000000003fffffff/30: 0xffffffff80000000
-> 0xffffffffbfffffff, 1 table(s)
domainbuilder: detail: nr_page_tables: 0x00000000001fffff/21: 0xffffffff80000000
-> 0xffffffff827fffff, 20 table(s)
domainbuilder: detail: xc_dom_alloc_segment: page tables : 0xffffffff824cf000
-> 0xffffffff824e6000 (pfn 0x24cf + 0x17 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x24cf+0x17 at
0x7fdece676000
domainbuilder: detail: xc_dom_alloc_page : boot stack : 0xffffffff824e6000
(pfn 0x24e6)
domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0xffffffff824e7000
domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0xffffffff82800000
domainbuilder: detail: xc_dom_boot_image: called
domainbuilder: detail: arch_setup_bootearly: doing nothing
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64
<= matches
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64
domainbuilder: detail: xc_dom_update_guest_p2m: dst 64bit, pages 0x138800
domainbuilder: detail: clear_page: pfn 0x24ce, mfn 0x37ddee
domainbuilder: detail: clear_page: pfn 0x24cd, mfn 0x37ddef
domainbuilder: detail: xc_dom_pfn_to_ptr: domU mapping: pfn 0x24cc+0x1 at
0x7fdece675000
domainbuilder: detail: start_info_x86_64: called
domainbuilder: detail: setup_hypercall_page: vaddr=0xffffffff81001000 pfn=0x1001
domainbuilder: detail: domain builder memory footprint
domainbuilder: detail: allocated
domainbuilder: detail: malloc : 20658 kB
domainbuilder: detail: anon mmap : 0 bytes
domainbuilder: detail: mapped
domainbuilder: detail: file mmap : 0 bytes
domainbuilder: detail: domU mmap : 21392 kB
domainbuilder: detail: arch_setup_bootlate: shared_info: pfn 0x0, mfn 0xbaa6f
domainbuilder: detail: shared_info_x86_64: called
domainbuilder: detail: vcpu_x86_64: called
domainbuilder: detail: vcpu_x86_64: cr3: pfn 0x24cf mfn 0x37dded
domainbuilder: detail: launch_vm: called, ctxt=0x7fff224e4ea0
domainbuilder: detail: xc_dom_release: called
Daemon running with PID 4639
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version 3.5.7-gentoo (root@majordomo) (gcc version 4.5.4
(Gentoo 4.5.4 p1.0, pie-0.4.7) ) #1 SMP Tue Nov 20 10:49:51 CET 2012
[ 0.000000] Command line: root=/dev/xvda1 console=hvc0 root=/dev/xvda1 ro 3
[ 0.000000] ACPI in unprivileged domain disabled
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
[ 0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
[ 0.000000] Xen: [mem 0x0000000000100000-0x0000000138ffffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] MPS support code is not built-in.
[ 0.000000] Using acpi=off or acpi=noirq or pci=noacpi may have problem
[ 0.000000] DMI not present or invalid.
[ 0.000000] No AGP bridge found
[ 0.000000] e820: last_pfn = 0x139000 max_arch_pfn = 0x400000000
[ 0.000000] e820: last_pfn = 0x100000 max_arch_pfn = 0x400000000
[ 0.000000] init_memory_mapping: [mem 0x00000000-0xffffffff]
[ 0.000000] init_memory_mapping: [mem 0x100000000-0x138ffffff]
[ 0.000000] NUMA turned off
[ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000138ffffff]
[ 0.000000] Initmem setup node 0 [mem 0x00000000-0x138ffffff]
[ 0.000000] NODE_DATA [mem 0x1387fc000-0x1387fffff]
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x00010000-0x00ffffff]
[ 0.000000] DMA32 [mem 0x01000000-0xffffffff]
[ 0.000000] Normal [mem 0x100000000-0x138ffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x00010000-0x0009ffff]
[ 0.000000] node 0: [mem 0x00100000-0x138ffffff]
[ 0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs
[ 0.000000] No local APIC present
[ 0.000000] APIC: disable apic facility
[ 0.000000] APIC: switched to apic NOOP
[ 0.000000] e820: cannot find a gap in the 32bit address range
[ 0.000000] e820: PCI devices with unassigned 32bit BARs may break!
[ 0.000000] e820: [mem 0x139100000-0x1394fffff] available for PCI devices
[ 0.000000] Booting paravirtualized kernel on Xen
[ 0.000000] Xen version: 4.1.1 (preserve-AD)
[ 0.000000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:1
nr_node_ids:1
[ 0.000000] PERCPU: Embedded 26 pages/cpu @ffff880138400000 s75712 r8192
d22592 u2097152
[ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total
pages: 1259871
[ 0.000000] Policy zone: Normal
[ 0.000000] Kernel command line: root=/dev/xvda1 console=hvc0 root=/dev/xvda1
ro 3
[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.000000] __ex_table already sorted, skipping sort
[ 0.000000] Checking aperture...
[ 0.000000] No AGP bridge found
[ 0.000000] Memory: 4943980k/5128192k available (3937k kernel code, 448k
absent, 183764k reserved, 1951k data, 524k init)
[ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=1,
Nodes=1
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] NR_IRQS:4352 nr_irqs:256 16
[ 0.000000] Console: colour dummy device 80x25
[ 0.000000] console [tty0] enabled
[ 0.000000] console [hvc0] enabled
[ 0.000000] installing Xen timer for CPU 0
[ 0.000000] Detected 3411.602 MHz processor.
[ 0.000999] Calibrating delay loop (skipped), value calculated using timer
frequency.. 6823.20 BogoMIPS (lpj=3411602)
[ 0.000999] pid_max: default: 32768 minimum: 301
[ 0.000999] Security Framework initialized
[ 0.001355] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[ 0.002974] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[ 0.003441] Mount-cache hash table entries: 256
[ 0.003595] Initializing cgroup subsys cpuacct
[ 0.003599] Initializing cgroup subsys freezer
[ 0.003637] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[ 0.003637] ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
[ 0.003643] CPU: Physical Processor ID: 0
[ 0.003645] CPU: Processor Core ID: 0
[ 0.003702] SMP alternatives: switching to UP code
[ 0.011791] Freeing SMP alternatives: 12k freed
[ 0.011835] Performance Events: unsupported p6 CPU model 42 no PMU driver,
software events only.
[ 0.011886] Brought up 1 CPUs
[ 0.011998] Grant tables using version 2 layout.
[ 0.012009] Grant table initialized
[ 0.012034] NET: Registered protocol family 16
[ 0.012328] PCI: setting up Xen PCI frontend stub
[ 0.015089] bio: create slab <bio-0> at 0
[ 0.015158] ACPI: Interpreter disabled.
[ 0.015180] xen/balloon: Initialising balloon driver.
[ 0.015180] xen-balloon: Initialising balloon driver.
[ 0.015180] vgaarb: loaded
[ 0.016126] SCSI subsystem initialized
[ 0.016314] PCI: System does not support PCI
[ 0.016320] PCI: System does not support PCI
[ 0.016435] NetLabel: Initializing
[ 0.016438] NetLabel: domain hash size = 128
[ 0.016440] NetLabel: protocols = UNLABELED CIPSOv4
[ 0.016447] NetLabel: unlabeled traffic allowed by default
[ 0.016475] Switching to clocksource xen
[ 0.017434] pnp: PnP ACPI: disabled
[ 0.017501] NET: Registered protocol family 2
[ 0.017864] IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
[ 0.019322] TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
[ 0.020376] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[ 0.020497] TCP: Hash tables configured (established 524288 bind 65536)
[ 0.020500] TCP: reno registered
[ 0.020525] UDP hash table entries: 4096 (order: 5, 131072 bytes)
[ 0.020564] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes)
[ 0.020624] NET: Registered protocol family 1
[ 0.020658] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 0.020662] software IO TLB [mem 0xfb632000-0xff631fff] (64MB) mapped at
[ffff8800fb632000-ffff8800ff631fff]
[ 0.020750] platform rtc_cmos: registered platform RTC device (no PNP device
found)
[ 0.021378] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 0.023378] msgmni has been set to 9656
[ 0.023544] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[ 0.023549] io scheduler noop registered
[ 0.023551] io scheduler deadline registered
[ 0.023580] io scheduler cfq registered (default)
[ 0.023650] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[ 0.023845] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 0.024082] Non-volatile memory driver v1.3
[ 0.024085] Linux agpgart interface v0.103
[ 0.024207] Event-channel device installed.
[ 0.024265] [drm] Initialized drm 1.1.0 20060810
[ 0.024268] [drm:i915_init] *ERROR* drm/i915 can't work without intel_agp module!
[ 0.025145] brd: module loaded
[ 0.025565] loop: module loaded
[ 0.045646] Initialising Xen virtual ethernet driver.
[ 0.198264] i8042: PNP: No PS/2 controller found. Probing ports directly.
[ 0.199096] i8042: No controller found
[ 0.199139] mousedev: PS/2 mouse device common for all mice
[ 0.259303] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[ 0.259353] rtc_cmos: probe of rtc_cmos failed with error -38
[ 0.259440] md: raid1 personality registered for level 1
[ 0.259542] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[ 0.259732] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 0.259747] TCP: cubic registered
[ 0.259886] NET: Registered protocol family 10
[ 0.260031] ip6_tables: (C) 2000-2006 Netfilter Core Team
[ 0.260070] sit: IPv6 over IPv4 tunneling driver
[ 0.260194] NET: Registered protocol family 17
[ 0.260213] Bridge firewalling registered
[ 5.360075] XENBUS: Waiting for devices to initialise:
25s...20s...15s...10s...5s...0s...235s...230s...225s...220s...215s...210s...205s...200s...195s...190s...185s...180s...175s...170s...165s...160s...155s...150s...145s...140s...135s...130s...125s...120s...115s...110s...105s...100s...95s...90s...85s...80s...75s...70s...65s...60s...55s...50s...45s...40s...35s...30s...25s...20s...15s...10s...5s...0s...
[ 270.360180] XENBUS: Timeout connecting to device: device/vbd/51713 (local
state 3, remote state 1)
[ 270.360273] md: Waiting for all devices to be available before autodetect
[ 270.360277] md: If you don't use raid, use raid=noautodetect
[ 270.360388] md: Autodetecting RAID arrays.
[ 270.360392] md: Scanned 0 and added 0 devices.
[ 270.360394] md: autorun ...
[ 270.360395] md: ... autorun DONE.
[ 270.360431] VFS: Cannot open root device "xvda1" or unknown-block(0,0): error -6
[ 270.360435] Please append a correct "root=" boot option; here are the
available partitions:
[ 270.360440] Kernel panic - not syncing: VFS: Unable to mount root fs on
unknown-block(0,0)
[ 270.360444] Pid: 1, comm: swapper/0 Not tainted 3.5.7-gentoo #1
[ 270.360446] Call Trace:
[ 270.360454] [<ffffffff813d2205>] ? panic+0xbe/0x1c5
[ 270.360459] [<ffffffff813d2358>] ? printk+0x4c/0x51
[ 270.360464] [<ffffffff815d5fb7>] ? mount_block_root+0x24f/0x26d
[ 270.360469] [<ffffffff815d62b6>] ? prepare_namespace+0x168/0x192
[ 270.360474] [<ffffffff815d5ca7>] ? kernel_init+0x1b0/0x1c2
[ 270.360477] [<ffffffff815d5500>] ? loglevel+0x34/0x34
[ 270.360482] [<ffffffff813d5a64>] ? kernel_thread_helper+0x4/0x10
[ 270.360486] [<ffffffff813d4038>] ? retint_restore_args+0x5/0x6
[ 270.360490] [<ffffffff813d5a60>] ? gs_change+0x13/0x13
The config:
name = "dom1"
bootloader = "/usr/bin/pygrub"
root = "/dev/xvda1 ro"
extra = "3" # runlevel
memory = 5000
disk = [ 'phy:/dev/md1,xvda1,w' ]
# vif = [ 'ip=..., vifname=veth1' ] # none for now
Here are some details on the Dom0 kernel (grepping for "xen"):
CONFIG_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PVHVM=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=500
CONFIG_XEN_SAVE_RESTORE=y
CONFIG_PCI_XEN=y
CONFIG_XEN_PCIDEV_FRONTEND=y
# CONFIG_XEN_BLKDEV_FRONTEND is not set
CONFIG_XEN_BLKDEV_BACKEND=y
# CONFIG_XEN_NETDEV_FRONTEND is not set
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y
CONFIG_HVC_XEN=y
CONFIG_HVC_XEN_FRONTEND=y
# CONFIG_XEN_WDT is not set
# CONFIG_XEN_FBDEV_FRONTEND is not set
# Xen driver support
CONFIG_XEN_BALLOON=y
# CONFIG_XEN_SELFBALLOONING is not set
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XEN_BACKEND=y
CONFIG_XENFS=y
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_GNTDEV=m
CONFIG_XEN_GRANT_DEV_ALLOC=m
CONFIG_SWIOTLB_XEN=y
CONFIG_XEN_TMEM=y
CONFIG_XEN_PCIDEV_BACKEND=m
CONFIG_XEN_PRIVCMD=y
CONFIG_XEN_ACPI_PROCESSOR=m
And the DomU kernel (grepping for "xen"):
CONFIG_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PVHVM=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=500
CONFIG_XEN_SAVE_RESTORE=y
CONFIG_PCI_XEN=y
CONFIG_XEN_PCIDEV_FRONTEND=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y
CONFIG_HVC_XEN=y
CONFIG_HVC_XEN_FRONTEND=y
# CONFIG_XEN_WDT is not set
# CONFIG_XEN_FBDEV_FRONTEND is not set
# Xen driver support
CONFIG_XEN_BALLOON=y
# CONFIG_XEN_SELFBALLOONING is not set
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=y
# CONFIG_XEN_BACKEND is not set
CONFIG_XENFS=y
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_GNTDEV=m
CONFIG_XEN_GRANT_DEV_ALLOC=m
CONFIG_SWIOTLB_XEN=y
CONFIG_XEN_TMEM=y
CONFIG_XEN_PRIVCMD=y
CONFIG_XEN_ACPI_PROCESSOR=m
Any ideas what I'm doing wrong here?
Thanks a lot!
