Discussion:
[PATCH V7 06/11] pci, acpi: Provide a way to assign bus domain number.
(too old to reply)
Tomasz Nowicki
2016-05-10 15:30:01 UTC
Permalink
This patch provides a way to set the ACPI domain number in PCI code.
pci_create_root_bus is called with NULL as parent in ACPI. This
ends up calling pci_bus_assign_domain_nr with a NULL parent.
So we define acpi_pci_bus_domain_nr() which is meant to retrieve
the PCI domain number based on 'struct pci_bus' in the ACPI way.
pci_bus_assign_domain_nr() is updated to call acpi_pci_bus_domain_nr()
and assign domain number on the root bus, in case of ACPI.

acpi_pci_bus_domain_nr function is stub for now.

While at it, for the sake of code clarity we put ACPI and DT domain
assign methods into the corresponding helpers.

Signed-off-by: Jayachandran C <***@broadcom.com>
Signed-off-by: Tomasz Nowicki <***@semihalf.com>
---
drivers/pci/pci.c | 11 +++++++++--
include/linux/pci-acpi.h | 5 +++++
2 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index ff97a0b..a1d7bcf 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -19,6 +19,7 @@
#include <linux/spinlock.h>
#include <linux/string.h>
#include <linux/log2.h>
+#include <linux/pci-acpi.h>
#include <linux/pci-aspm.h>
#include <linux/pm_wakeup.h>
#include <linux/interrupt.h>
@@ -4918,7 +4919,7 @@ int pci_get_new_domain_nr(void)
}

#ifdef CONFIG_PCI_DOMAINS_GENERIC
-void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
+static int of_pci_bus_domain_nr(struct device *parent)
{
static int use_dt_domains = -1;
int domain = -1;
@@ -4962,7 +4963,13 @@ void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
domain = -1;
}

- bus->domain_nr = domain;
+ return domain;
+}
+
+void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent)
+{
+ bus->domain_nr = acpi_disabled ? of_pci_bus_domain_nr(parent) :
+ acpi_pci_bus_domain_nr(bus);
}
#endif
#endif
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 89ab057..09f9f02 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -111,6 +111,11 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
#endif /* CONFIG_ACPI */

+static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus)
+{
+ return 0;
+}
+
#ifdef CONFIG_ACPI_APEI
extern bool aer_acpi_firmware_first(void);
#else
--
1.9.1
Tomasz Nowicki
2016-05-10 15:30:01 UTC
Permalink
To enable PCI legacy IRQs on platforms booting with ACPI, arch code
should include ACPI specific callbacks that parse and set-up the
device IRQ number, equivalent to the DT boot path. Owing to the current
ACPI core scan handlers implementation, ACPI PCI legacy IRQs bindings
cannot be parsed at device add time, since that would trigger ACPI scan
handlers ordering issues depending on how the ACPI tables are defined.

To solve this problem and consolidate FW PCI legacy IRQs parsing in
one single pcibios callback (pending final removal), this patch moves
DT PCI IRQ parsing to the pcibios_alloc_irq() callback (called by
PCI core code at device probe time) and adds ACPI PCI legacy IRQs
parsing to the same callback too, so that FW PCI legacy IRQs parsing
is confined in one single arch callback that can be easily removed
when code parsing PCI legacy IRQs is consolidated and moved to core
PCI code.

Signed-off-by: Tomasz Nowicki <***@semihalf.com>
Suggested-by: Lorenzo Pieralisi <***@arm.com>
---
arch/arm64/kernel/pci.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index c72de66..15109c11 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -50,11 +50,16 @@ int pcibios_enable_device(struct pci_dev *dev, int mask)
}

/*
- * Try to assign the IRQ number from DT when adding a new device
+ * Try to assign the IRQ number when probing a new device
*/
-int pcibios_add_device(struct pci_dev *dev)
+int pcibios_alloc_irq(struct pci_dev *dev)
{
- dev->irq = of_irq_parse_and_map_pci(dev, 0, 0);
+ if (acpi_disabled)
+ dev->irq = of_irq_parse_and_map_pci(dev, 0, 0);
+#ifdef CONFIG_ACPI
+ else
+ return acpi_pci_irq_enable(dev);
+#endif

return 0;
}
--
1.9.1
Tomasz Nowicki
2016-05-10 15:30:02 UTC
Permalink
ACPI requires to run acpi_pci_{add|remove}_bus while new PCI bus is created.
This allows to do some ACPI-specific additional configuration, like
PCI hotplug slot enumeration. In order to fulfill these requirements,
we implement arch-specific pcibios_{add|remove}_bus calls
and call acpi_pci_{add|remove}_bus from there.

Signed-off-by: Tomasz Nowicki <***@semihalf.com>
To: Catalin Marinas <***@arm.com>
To: Lorenzo Pieralisi <***@arm.com>
To: Will Deacon <***@arm.com>
To: Arnd Bergmann <***@arndb.de>
---
arch/arm64/kernel/pci.c | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index 15109c11..eeec5f6 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -17,6 +17,7 @@
#include <linux/mm.h>
#include <linux/of_pci.h>
#include <linux/of_platform.h>
+#include <linux/pci-acpi.h>
#include <linux/slab.h>

/*
@@ -64,6 +65,16 @@ int pcibios_alloc_irq(struct pci_dev *dev)
return 0;
}

+void pcibios_add_bus(struct pci_bus *bus)
+{
+ acpi_pci_add_bus(bus);
+}
+
+void pcibios_remove_bus(struct pci_bus *bus)
+{
+ acpi_pci_remove_bus(bus);
+}
+
/*
* raw_pci_read/write - Platform-specific PCI config space access.
*/
--
1.9.1
Tomasz Nowicki
2016-05-10 15:30:02 UTC
Permalink
This patch provides a way to set the ACPI companion in PCI code.
We define acpi_pci_set_companion() to set the ACPI companion pointer and
call it from PCI core code. The function is stub for now.

Signed-off-by: Jayachandran C <***@broadcom.com>
Signed-off-by: Tomasz Nowicki <***@semihalf.com>
---
drivers/pci/probe.c | 2 ++
include/linux/pci-acpi.h | 4 ++++
2 files changed, 6 insertions(+)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8004f67..fb0b752 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -12,6 +12,7 @@
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/cpumask.h>
+#include <linux/pci-acpi.h>
#include <linux/pci-aspm.h>
#include <linux/aer.h>
#include <linux/acpi.h>
@@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
bridge->dev.parent = parent;
bridge->dev.release = pci_release_host_bridge_dev;
dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
+ acpi_pci_set_companion(bridge);
error = pcibios_root_bridge_prepare(bridge);
if (error) {
kfree(bridge);
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 09f9f02..1baa515 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
#endif /* CONFIG_ACPI */

+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge)
+{
+}
+
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus)
{
return 0;
--
1.9.1
Rafael J. Wysocki
2016-05-10 18:40:01 UTC
Permalink
Post by Tomasz Nowicki
This patch provides a way to set the ACPI companion in PCI code.
We define acpi_pci_set_companion() to set the ACPI companion pointer and
call it from PCI core code. The function is stub for now.
---
drivers/pci/probe.c | 2 ++
include/linux/pci-acpi.h | 4 ++++
2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8004f67..fb0b752 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -12,6 +12,7 @@
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/cpumask.h>
+#include <linux/pci-acpi.h>
#include <linux/pci-aspm.h>
#include <linux/aer.h>
#include <linux/acpi.h>
@@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
bridge->dev.parent = parent;
bridge->dev.release = pci_release_host_bridge_dev;
dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
+ acpi_pci_set_companion(bridge);
Yes, we'll probably add something similar here.

Do I think now is the right time to do that? No.
Post by Tomasz Nowicki
error = pcibios_root_bridge_prepare(bridge);
if (error) {
kfree(bridge);
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 09f9f02..1baa515 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
#endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge)
+{
+}
+
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus)
{
return 0;
--
Honestly, to me it looks like this series is trying very hard to avoid
doing any PCI host bridge configuration stuff from arch/arm64/
although (a) that might be simpler and (b) it would allow us to
identify the code that's common between *all* architectures using ACPI
support for host bridge configuration and to move *that* to a common
place later. As done here it seems to be following the "ARM64 is
generic and the rest of the world is special" line which isn't really
helpful.
Rafael J. Wysocki
2016-05-10 18:50:03 UTC
Permalink
Post by Rafael J. Wysocki
Post by Tomasz Nowicki
This patch provides a way to set the ACPI companion in PCI code.
We define acpi_pci_set_companion() to set the ACPI companion pointer and
call it from PCI core code. The function is stub for now.
---
drivers/pci/probe.c | 2 ++
include/linux/pci-acpi.h | 4 ++++
2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8004f67..fb0b752 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -12,6 +12,7 @@
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/cpumask.h>
+#include <linux/pci-acpi.h>
#include <linux/pci-aspm.h>
#include <linux/aer.h>
#include <linux/acpi.h>
@@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
bridge->dev.parent = parent;
bridge->dev.release = pci_release_host_bridge_dev;
dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
+ acpi_pci_set_companion(bridge);
Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
Post by Tomasz Nowicki
error = pcibios_root_bridge_prepare(bridge);
if (error) {
kfree(bridge);
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 09f9f02..1baa515 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
#endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge)
+{
+}
+
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus)
{
return 0;
--
Honestly, to me it looks like this series is trying very hard to avoid
doing any PCI host bridge configuration stuff from arch/arm64/
although (a) that might be simpler and (b) it would allow us to
identify the code that's common between *all* architectures using ACPI
support for host bridge configuration and to move *that* to a common
place later. As done here it seems to be following the "ARM64 is
generic and the rest of the world is special" line which isn't really
helpful.
Speaking of which, at least one of the reasons why the ACPI PCI host
bridge thing on x86 and ia64 went to the arch code was to avoid
explicit references to ACPI-specific data types and related #ifdeffery
in the generic PCI code and data structures. If you are going to add
those references now anyway, that reason is not relevant any more and
all of that can just be reworked to refer to ACPI explicitly.
Lorenzo Pieralisi
2016-05-11 10:20:03 UTC
Permalink
Post by Rafael J. Wysocki
Post by Tomasz Nowicki
This patch provides a way to set the ACPI companion in PCI code.
We define acpi_pci_set_companion() to set the ACPI companion pointer and
call it from PCI core code. The function is stub for now.
---
drivers/pci/probe.c | 2 ++
include/linux/pci-acpi.h | 4 ++++
2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8004f67..fb0b752 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -12,6 +12,7 @@
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/cpumask.h>
+#include <linux/pci-acpi.h>
#include <linux/pci-aspm.h>
#include <linux/aer.h>
#include <linux/acpi.h>
@@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
bridge->dev.parent = parent;
bridge->dev.release = pci_release_host_bridge_dev;
dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
+ acpi_pci_set_companion(bridge);
Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
Post by Tomasz Nowicki
error = pcibios_root_bridge_prepare(bridge);
if (error) {
kfree(bridge);
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 09f9f02..1baa515 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
#endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge)
+{
+}
+
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus)
{
return 0;
--
Honestly, to me it looks like this series is trying very hard to avoid
doing any PCI host bridge configuration stuff from arch/arm64/
although (a) that might be simpler and (b) it would allow us to
identify the code that's common between *all* architectures using ACPI
support for host bridge configuration and to move *that* to a common
place later. As done here it seems to be following the "ARM64 is
generic and the rest of the world is special" line which isn't really
helpful.
I think patch [1-2] should be merged regardless (they may require minor
tweaks if we decide to move pci_acpi_scan_root() to arch/arm64 though,
for include files location). I guess you are referring to patch 8 in
your comments above, which boils down to deciding whether:

- pci_acpi_scan_root() (and unfortunately all the MCFG/ECAM handling that
goes with it) should live in arch/arm64 or drivers/acpi

acpi_pci_bus_domain_nr() is a bit more problematic since it is meant
to be called from PCI core code (ARM64 selects PCI_DOMAINS_GENERIC for
DT and same kernel has to work with OF and ACPI selected) and it is
arch specific (because what we have in bus->sysdata is arch specific,
waiting for the domain number to be embedded in struct pci_host_bridge).

Your point is fair, I am not sure that moving the pci_acpi_scan_root()
to arch/arm64 would make things much simpler though, it is just a matter
of deciding where that code has to live.

How do you want us to proceed ?

Thanks,
Lorenzo
Rafael J. Wysocki
2016-05-11 20:40:01 UTC
Permalink
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi
Post by Lorenzo Pieralisi
Post by Rafael J. Wysocki
Post by Tomasz Nowicki
This patch provides a way to set the ACPI companion in PCI code.
We define acpi_pci_set_companion() to set the ACPI companion pointer and
call it from PCI core code. The function is stub for now.
---
drivers/pci/probe.c | 2 ++
include/linux/pci-acpi.h | 4 ++++
2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8004f67..fb0b752 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -12,6 +12,7 @@
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/cpumask.h>
+#include <linux/pci-acpi.h>
#include <linux/pci-aspm.h>
#include <linux/aer.h>
#include <linux/acpi.h>
@@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
bridge->dev.parent = parent;
bridge->dev.release = pci_release_host_bridge_dev;
dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
+ acpi_pci_set_companion(bridge);
Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
Post by Tomasz Nowicki
error = pcibios_root_bridge_prepare(bridge);
if (error) {
kfree(bridge);
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 09f9f02..1baa515 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
#endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge)
+{
+}
+
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus)
{
return 0;
--
Honestly, to me it looks like this series is trying very hard to avoid
doing any PCI host bridge configuration stuff from arch/arm64/
although (a) that might be simpler and (b) it would allow us to
identify the code that's common between *all* architectures using ACPI
support for host bridge configuration and to move *that* to a common
place later. As done here it seems to be following the "ARM64 is
generic and the rest of the world is special" line which isn't really
helpful.
I think patch [1-2] should be merged regardless (they may require minor
tweaks if we decide to move pci_acpi_scan_root() to arch/arm64 though,
for include files location). I guess you are referring to patch 8 in
- pci_acpi_scan_root() (and unfortunately all the MCFG/ECAM handling that
goes with it) should live in arch/arm64 or drivers/acpi
To be precise, everything under #ifdef CONFIG_ACPI_PCI_HOST_GENERIC or
equivalent is de facto ARM64-specific, because (as it stands in the
patch series) ARM64 is the only architecture that will select that
option. Unless you are aware of any more architectures planning to
use ACPI (and I'm not aware of any), it will stay the only
architecture selecting it in the foreseeable future.

Therefore you could replace CONFIG_ACPI_PCI_HOST_GENERIC with
CONFIG_ARM64 everywhere in that code which is why in my opinion the
code should live somewhere under arch/arm64/.

Going forward, it should be possible to identify common parts of the
PCI host bridge configuration code in arch/ and move it to
drivers/acpi/ or drivers/pci/, but I bet that won't be the entire code
this series puts under CONFIG_ACPI_PCI_HOST_GENERIC.

The above leads to a quite straightforward conclusion about the order
in which to do things: I'd add ACPI support for PCI host bridge on
ARM64 following what's been done on ia64 (as x86 is more quirky and
kludgy overall) as far as reasonably possible first and then think
about moving common stuff to a common place.
Post by Lorenzo Pieralisi
acpi_pci_bus_domain_nr() is a bit more problematic since it is meant
to be called from PCI core code (ARM64 selects PCI_DOMAINS_GENERIC for
DT and same kernel has to work with OF and ACPI selected) and it is
arch specific (because what we have in bus->sysdata is arch specific,
waiting for the domain number to be embedded in struct pci_host_bridge).
Your point is fair, I am not sure that moving the pci_acpi_scan_root()
to arch/arm64 would make things much simpler though, it is just a matter
of deciding where that code has to live.
How do you want us to proceed ?
Pretty much as stated above. :-)

Thanks,
Rafael
Bjorn Helgaas
2016-05-11 22:50:01 UTC
Permalink
Post by Rafael J. Wysocki
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi
Post by Lorenzo Pieralisi
Post by Rafael J. Wysocki
Post by Tomasz Nowicki
This patch provides a way to set the ACPI companion in PCI code.
We define acpi_pci_set_companion() to set the ACPI companion pointer and
call it from PCI core code. The function is stub for now.
---
drivers/pci/probe.c | 2 ++
include/linux/pci-acpi.h | 4 ++++
2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8004f67..fb0b752 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -12,6 +12,7 @@
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/cpumask.h>
+#include <linux/pci-acpi.h>
#include <linux/pci-aspm.h>
#include <linux/aer.h>
#include <linux/acpi.h>
@@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
bridge->dev.parent = parent;
bridge->dev.release = pci_release_host_bridge_dev;
dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
+ acpi_pci_set_companion(bridge);
Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
Post by Tomasz Nowicki
error = pcibios_root_bridge_prepare(bridge);
if (error) {
kfree(bridge);
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 09f9f02..1baa515 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
#endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge)
+{
+}
+
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus)
{
return 0;
--
Honestly, to me it looks like this series is trying very hard to avoid
doing any PCI host bridge configuration stuff from arch/arm64/
although (a) that might be simpler and (b) it would allow us to
identify the code that's common between *all* architectures using ACPI
support for host bridge configuration and to move *that* to a common
place later. As done here it seems to be following the "ARM64 is
generic and the rest of the world is special" line which isn't really
helpful.
I think patch [1-2] should be merged regardless (they may require minor
tweaks if we decide to move pci_acpi_scan_root() to arch/arm64 though,
for include files location). I guess you are referring to patch 8 in
- pci_acpi_scan_root() (and unfortunately all the MCFG/ECAM handling that
goes with it) should live in arch/arm64 or drivers/acpi
To be precise, everything under #ifdef CONFIG_ACPI_PCI_HOST_GENERIC or
equivalent is de facto ARM64-specific, because (as it stands in the
patch series) ARM64 is the only architecture that will select that
option. Unless you are aware of any more architectures planning to
use ACPI (and I'm not aware of any), it will stay the only
architecture selecting it in the foreseeable future.
Therefore you could replace CONFIG_ACPI_PCI_HOST_GENERIC with
CONFIG_ARM64 everywhere in that code which is why in my opinion the
code should live somewhere under arch/arm64/.
Going forward, it should be possible to identify common parts of the
PCI host bridge configuration code in arch/ and move it to
drivers/acpi/ or drivers/pci/, but I bet that won't be the entire code
this series puts under CONFIG_ACPI_PCI_HOST_GENERIC.
The above leads to a quite straightforward conclusion about the order
in which to do things: I'd add ACPI support for PCI host bridge on
ARM64 following what's been done on ia64 (as x86 is more quirky and
kludgy overall) as far as reasonably possible first and then think
about moving common stuff to a common place.
That does seem like a reasonable approach. I had hoped to get more of
this in for v4.7, but we don't have much time left. Maybe some of
Rafael's comments can be addressed by moving and slight restructuring
and we can still squeeze it in.

The first three patches:

PCI: Provide common functions for ECAM mapping
PCI: generic, thunder: Use generic ECAM API
PCI, of: Move PCI I/O space management to PCI core code

seem relatively straightforward, and I applied them to pci/arm64 with
the intent of merging them unless there are objections. I made the
following tweaks, mainly to try to improve some error messages:

diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
index 3d52005..e1add01 100644
--- a/drivers/pci/ecam.c
+++ b/drivers/pci/ecam.c
@@ -24,9 +24,9 @@
#include "ecam.h"

/*
- * On 64 bit systems, we do a single ioremap for the whole config space
- * since we have enough virtual address range available. On 32 bit, do an
- * ioremap per bus.
+ * On 64-bit systems, we do a single ioremap for the whole config space
+ * since we have enough virtual address range available. On 32-bit, we
+ * ioremap the config space for each bus individually.
*/
static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);

@@ -42,6 +42,7 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
{
struct pci_config_window *cfg;
unsigned int bus_range, bus_range_max, bsz;
+ struct resource *conflict;
int i, err;

if (busr->start > busr->end)
@@ -58,10 +59,10 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
bus_range = resource_size(&cfg->busr);
bus_range_max = resource_size(cfgres) >> ops->bus_shift;
if (bus_range > bus_range_max) {
- dev_warn(dev, "bus max %#x reduced to %#x",
- bus_range, bus_range_max);
bus_range = bus_range_max;
cfg->busr.end = busr->start + bus_range - 1;
+ dev_warn(dev, "ECAM area %pR can only accommodate %pR (reduced from %pR desired)\n",
+ cfgres, &cfg->busr, busr);
}
bsz = 1 << ops->bus_shift;

@@ -70,9 +71,11 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
cfg->res.flags = IORESOURCE_MEM | IORESOURCE_BUSY;
cfg->res.name = "PCI ECAM";

- err = request_resource(&iomem_resource, &cfg->res);
- if (err) {
- dev_err(dev, "request ECAM res %pR failed\n", &cfg->res);
+ conflict = request_resource(&iomem_resource, &cfg->res);
+ if (conflict) {
+ err = -EBUSY;
+ dev_err(dev, "can't claim ECAM area %pR: address conflict with %s %pR\n",
+ &cfg->res, conflict->name, conflict);
goto err_exit;
}

diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
index 1ad2176..9878beb 100644
--- a/drivers/pci/ecam.h
+++ b/drivers/pci/ecam.h
@@ -33,7 +33,7 @@ struct pci_ecam_ops {

/*
* struct to hold the mappings of a config space window. This
- * is expected to be used as sysdata for PCI controlllers which
+ * is expected to be used as sysdata for PCI controllers that
* use ECAM.
*/
struct pci_config_window {
@@ -43,11 +43,11 @@ struct pci_config_window {
struct pci_ecam_ops *ops;
union {
void __iomem *win; /* 64-bit single mapping */
- void __iomem **winp; /* 32-bit per bus mapping */
+ void __iomem **winp; /* 32-bit per-bus mapping */
};
};

-/* create and free for pci_config_window */
+/* create and free pci_config_window */
struct pci_config_window *pci_ecam_create(struct device *dev,
struct resource *cfgres, struct resource *busr,
struct pci_ecam_ops *ops);
@@ -56,11 +56,11 @@ void pci_ecam_free(struct pci_config_window *cfg);
/* map_bus when ->sysdata is an instance of pci_config_window */
void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
int where);
-/* default ECAM ops, bus shift 20, generic read and write */
+/* default ECAM ops */
extern struct pci_ecam_ops pci_generic_ecam_ops;

#ifdef CONFIG_PCI_HOST_GENERIC
-/* for DT based pci controllers that support ECAM */
+/* for DT-based PCI controllers that support ECAM */
int pci_host_common_probe(struct platform_device *pdev,
struct pci_ecam_ops *ops);
#endif
Lorenzo Pieralisi
2016-05-12 10:10:02 UTC
Permalink
Post by Bjorn Helgaas
Post by Rafael J. Wysocki
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi
Post by Lorenzo Pieralisi
Post by Rafael J. Wysocki
Post by Tomasz Nowicki
This patch provides a way to set the ACPI companion in PCI code.
We define acpi_pci_set_companion() to set the ACPI companion pointer and
call it from PCI core code. The function is stub for now.
---
drivers/pci/probe.c | 2 ++
include/linux/pci-acpi.h | 4 ++++
2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8004f67..fb0b752 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -12,6 +12,7 @@
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/cpumask.h>
+#include <linux/pci-acpi.h>
#include <linux/pci-aspm.h>
#include <linux/aer.h>
#include <linux/acpi.h>
@@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
bridge->dev.parent = parent;
bridge->dev.release = pci_release_host_bridge_dev;
dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
+ acpi_pci_set_companion(bridge);
Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
Post by Tomasz Nowicki
error = pcibios_root_bridge_prepare(bridge);
if (error) {
kfree(bridge);
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 09f9f02..1baa515 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
#endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge)
+{
+}
+
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus)
{
return 0;
--
Honestly, to me it looks like this series is trying very hard to avoid
doing any PCI host bridge configuration stuff from arch/arm64/
although (a) that might be simpler and (b) it would allow us to
identify the code that's common between *all* architectures using ACPI
support for host bridge configuration and to move *that* to a common
place later. As done here it seems to be following the "ARM64 is
generic and the rest of the world is special" line which isn't really
helpful.
I think patch [1-2] should be merged regardless (they may require minor
tweaks if we decide to move pci_acpi_scan_root() to arch/arm64 though,
for include files location). I guess you are referring to patch 8 in
- pci_acpi_scan_root() (and unfortunately all the MCFG/ECAM handling that
goes with it) should live in arch/arm64 or drivers/acpi
To be precise, everything under #ifdef CONFIG_ACPI_PCI_HOST_GENERIC or
equivalent is de facto ARM64-specific, because (as it stands in the
patch series) ARM64 is the only architecture that will select that
option. Unless you are aware of any more architectures planning to
use ACPI (and I'm not aware of any), it will stay the only
architecture selecting it in the foreseeable future.
Therefore you could replace CONFIG_ACPI_PCI_HOST_GENERIC with
CONFIG_ARM64 everywhere in that code which is why in my opinion the
code should live somewhere under arch/arm64/.
Going forward, it should be possible to identify common parts of the
PCI host bridge configuration code in arch/ and move it to
drivers/acpi/ or drivers/pci/, but I bet that won't be the entire code
this series puts under CONFIG_ACPI_PCI_HOST_GENERIC.
The above leads to a quite straightforward conclusion about the order
in which to do things: I'd add ACPI support for PCI host bridge on
ARM64 following what's been done on ia64 (as x86 is more quirky and
kludgy overall) as far as reasonably possible first and then think
about moving common stuff to a common place.
That does seem like a reasonable approach. I had hoped to get more of
this in for v4.7, but we don't have much time left. Maybe some of
Rafael's comments can be addressed by moving and slight restructuring
and we can still squeeze it in.
Yes, it seems like a reasonable approach, as long as we accept that
part of this series has to live in arch/arm64 otherwise we are going
round in circles (because that's the gist of this discussion, to
decide where this code has to live, I do not think there is any objection
to the code per-se anymore).

I suggest we post a v8 (with code move to arch/arm64) end of merge
window (or you prefer seeing patches now to prevent any additional
changes later ?), my aim is to get this into -next (whether via arm64 or
pci tree it has to be decided) as early as possible for next cycle (-rc1)
so that it can get exposure and testing, I do not think that missing the
merge window is a big issue if we agree that the code is ready to go.
Post by Bjorn Helgaas
PCI: Provide common functions for ECAM mapping
PCI: generic, thunder: Use generic ECAM API
PCI, of: Move PCI I/O space management to PCI core code
seem relatively straightforward, and I applied them to pci/arm64 with
the intent of merging them unless there are objections. I made the
Ok, thanks a lot !

Lorenzo
Post by Bjorn Helgaas
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
index 3d52005..e1add01 100644
--- a/drivers/pci/ecam.c
+++ b/drivers/pci/ecam.c
@@ -24,9 +24,9 @@
#include "ecam.h"
/*
- * On 64 bit systems, we do a single ioremap for the whole config space
- * since we have enough virtual address range available. On 32 bit, do an
- * ioremap per bus.
+ * On 64-bit systems, we do a single ioremap for the whole config space
+ * since we have enough virtual address range available. On 32-bit, we
+ * ioremap the config space for each bus individually.
*/
static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);
@@ -42,6 +42,7 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
{
struct pci_config_window *cfg;
unsigned int bus_range, bus_range_max, bsz;
+ struct resource *conflict;
int i, err;
if (busr->start > busr->end)
@@ -58,10 +59,10 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
bus_range = resource_size(&cfg->busr);
bus_range_max = resource_size(cfgres) >> ops->bus_shift;
if (bus_range > bus_range_max) {
- dev_warn(dev, "bus max %#x reduced to %#x",
- bus_range, bus_range_max);
bus_range = bus_range_max;
cfg->busr.end = busr->start + bus_range - 1;
+ dev_warn(dev, "ECAM area %pR can only accommodate %pR (reduced from %pR desired)\n",
+ cfgres, &cfg->busr, busr);
}
bsz = 1 << ops->bus_shift;
@@ -70,9 +71,11 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
cfg->res.flags = IORESOURCE_MEM | IORESOURCE_BUSY;
cfg->res.name = "PCI ECAM";
- err = request_resource(&iomem_resource, &cfg->res);
- if (err) {
- dev_err(dev, "request ECAM res %pR failed\n", &cfg->res);
+ conflict = request_resource(&iomem_resource, &cfg->res);
+ if (conflict) {
+ err = -EBUSY;
+ dev_err(dev, "can't claim ECAM area %pR: address conflict with %s %pR\n",
+ &cfg->res, conflict->name, conflict);
goto err_exit;
}
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
index 1ad2176..9878beb 100644
--- a/drivers/pci/ecam.h
+++ b/drivers/pci/ecam.h
@@ -33,7 +33,7 @@ struct pci_ecam_ops {
/*
* struct to hold the mappings of a config space window. This
- * is expected to be used as sysdata for PCI controlllers which
+ * is expected to be used as sysdata for PCI controllers that
* use ECAM.
*/
struct pci_config_window {
@@ -43,11 +43,11 @@ struct pci_config_window {
struct pci_ecam_ops *ops;
union {
void __iomem *win; /* 64-bit single mapping */
- void __iomem **winp; /* 32-bit per bus mapping */
+ void __iomem **winp; /* 32-bit per-bus mapping */
};
};
-/* create and free for pci_config_window */
+/* create and free pci_config_window */
struct pci_config_window *pci_ecam_create(struct device *dev,
struct resource *cfgres, struct resource *busr,
struct pci_ecam_ops *ops);
@@ -56,11 +56,11 @@ void pci_ecam_free(struct pci_config_window *cfg);
/* map_bus when ->sysdata is an instance of pci_config_window */
void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
int where);
-/* default ECAM ops, bus shift 20, generic read and write */
+/* default ECAM ops */
extern struct pci_ecam_ops pci_generic_ecam_ops;
#ifdef CONFIG_PCI_HOST_GENERIC
-/* for DT based pci controllers that support ECAM */
+/* for DT-based PCI controllers that support ECAM */
int pci_host_common_probe(struct platform_device *pdev,
struct pci_ecam_ops *ops);
#endif
Jayachandran C
2016-05-12 10:50:02 UTC
Permalink
Post by Bjorn Helgaas
Post by Rafael J. Wysocki
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi
Post by Lorenzo Pieralisi
Post by Rafael J. Wysocki
Post by Tomasz Nowicki
This patch provides a way to set the ACPI companion in PCI code.
We define acpi_pci_set_companion() to set the ACPI companion pointer and
call it from PCI core code. The function is stub for now.
---
drivers/pci/probe.c | 2 ++
include/linux/pci-acpi.h | 4 ++++
2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8004f67..fb0b752 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -12,6 +12,7 @@
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/cpumask.h>
+#include <linux/pci-acpi.h>
#include <linux/pci-aspm.h>
#include <linux/aer.h>
#include <linux/acpi.h>
@@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
bridge->dev.parent = parent;
bridge->dev.release = pci_release_host_bridge_dev;
dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
+ acpi_pci_set_companion(bridge);
Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
Post by Tomasz Nowicki
error = pcibios_root_bridge_prepare(bridge);
if (error) {
kfree(bridge);
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 09f9f02..1baa515 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
#endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge)
+{
+}
+
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus)
{
return 0;
--
Honestly, to me it looks like this series is trying very hard to avoid
doing any PCI host bridge configuration stuff from arch/arm64/
although (a) that might be simpler and (b) it would allow us to
identify the code that's common between *all* architectures using ACPI
support for host bridge configuration and to move *that* to a common
place later. As done here it seems to be following the "ARM64 is
generic and the rest of the world is special" line which isn't really
helpful.
I think patch [1-2] should be merged regardless (they may require minor
tweaks if we decide to move pci_acpi_scan_root() to arch/arm64 though,
for include files location). I guess you are referring to patch 8 in
- pci_acpi_scan_root() (and unfortunately all the MCFG/ECAM handling that
goes with it) should live in arch/arm64 or drivers/acpi
To be precise, everything under #ifdef CONFIG_ACPI_PCI_HOST_GENERIC or
equivalent is de facto ARM64-specific, because (as it stands in the
patch series) ARM64 is the only architecture that will select that
option. Unless you are aware of any more architectures planning to
use ACPI (and I'm not aware of any), it will stay the only
architecture selecting it in the foreseeable future.
Therefore you could replace CONFIG_ACPI_PCI_HOST_GENERIC with
CONFIG_ARM64 everywhere in that code which is why in my opinion the
code should live somewhere under arch/arm64/.
Going forward, it should be possible to identify common parts of the
PCI host bridge configuration code in arch/ and move it to
drivers/acpi/ or drivers/pci/, but I bet that won't be the entire code
this series puts under CONFIG_ACPI_PCI_HOST_GENERIC.
The above leads to a quite straightforward conclusion about the order
in which to do things: I'd add ACPI support for PCI host bridge on
ARM64 following what's been done on ia64 (as x86 is more quirky and
kludgy overall) as far as reasonably possible first and then think
about moving common stuff to a common place.
That does seem like a reasonable approach. I had hoped to get more of
this in for v4.7, but we don't have much time left. Maybe some of
Rafael's comments can be addressed by moving and slight restructuring
and we can still squeeze it in.
PCI: Provide common functions for ECAM mapping
PCI: generic, thunder: Use generic ECAM API
PCI, of: Move PCI I/O space management to PCI core code
seem relatively straightforward, and I applied them to pci/arm64 with
the intent of merging them unless there are objections. I made the
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
index 3d52005..e1add01 100644
--- a/drivers/pci/ecam.c
+++ b/drivers/pci/ecam.c
@@ -24,9 +24,9 @@
#include "ecam.h"
/*
- * On 64 bit systems, we do a single ioremap for the whole config space
- * since we have enough virtual address range available. On 32 bit, do an
- * ioremap per bus.
+ * On 64-bit systems, we do a single ioremap for the whole config space
+ * since we have enough virtual address range available. On 32-bit, we
+ * ioremap the config space for each bus individually.
*/
static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);
@@ -42,6 +42,7 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
{
struct pci_config_window *cfg;
unsigned int bus_range, bus_range_max, bsz;
+ struct resource *conflict;
int i, err;
if (busr->start > busr->end)
@@ -58,10 +59,10 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
bus_range = resource_size(&cfg->busr);
bus_range_max = resource_size(cfgres) >> ops->bus_shift;
if (bus_range > bus_range_max) {
- dev_warn(dev, "bus max %#x reduced to %#x",
- bus_range, bus_range_max);
bus_range = bus_range_max;
cfg->busr.end = busr->start + bus_range - 1;
+ dev_warn(dev, "ECAM area %pR can only accommodate %pR (reduced from %pR desired)\n",
+ cfgres, &cfg->busr, busr);
}
bsz = 1 << ops->bus_shift;
@@ -70,9 +71,11 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
cfg->res.flags = IORESOURCE_MEM | IORESOURCE_BUSY;
cfg->res.name = "PCI ECAM";
- err = request_resource(&iomem_resource, &cfg->res);
- if (err) {
- dev_err(dev, "request ECAM res %pR failed\n", &cfg->res);
+ conflict = request_resource(&iomem_resource, &cfg->res);
+ if (conflict) {
+ err = -EBUSY;
+ dev_err(dev, "can't claim ECAM area %pR: address conflict with %s %pR\n",
+ &cfg->res, conflict->name, conflict);
goto err_exit;
}
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
index 1ad2176..9878beb 100644
--- a/drivers/pci/ecam.h
+++ b/drivers/pci/ecam.h
@@ -33,7 +33,7 @@ struct pci_ecam_ops {
/*
* struct to hold the mappings of a config space window. This
- * is expected to be used as sysdata for PCI controlllers which
+ * is expected to be used as sysdata for PCI controllers that
* use ECAM.
*/
struct pci_config_window {
@@ -43,11 +43,11 @@ struct pci_config_window {
struct pci_ecam_ops *ops;
union {
void __iomem *win; /* 64-bit single mapping */
- void __iomem **winp; /* 32-bit per bus mapping */
+ void __iomem **winp; /* 32-bit per-bus mapping */
};
};
-/* create and free for pci_config_window */
+/* create and free pci_config_window */
struct pci_config_window *pci_ecam_create(struct device *dev,
struct resource *cfgres, struct resource *busr,
struct pci_ecam_ops *ops);
@@ -56,11 +56,11 @@ void pci_ecam_free(struct pci_config_window *cfg);
/* map_bus when ->sysdata is an instance of pci_config_window */
void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
int where);
-/* default ECAM ops, bus shift 20, generic read and write */
+/* default ECAM ops */
extern struct pci_ecam_ops pci_generic_ecam_ops;
#ifdef CONFIG_PCI_HOST_GENERIC
-/* for DT based pci controllers that support ECAM */
+/* for DT-based PCI controllers that support ECAM */
int pci_host_common_probe(struct platform_device *pdev,
struct pci_ecam_ops *ops);
#endif
If we are moving the ACPI/PCI code from drivers/acpi to
arch/arm64/ , there is an issue in having the header file
ecam.h in drivers/pci

The current include of "../pci/ecam.h" is slightly ugly (Arnd
and David had already noted this), but including the driver
header from arch code would be even worse.

I can either merge ecam.h into include/linux/pci.h
or move it to a new file include/linux/pci-ecam.h, any
suggestion on which is preferable?

JC.
Rafael J. Wysocki
2016-05-12 11:30:03 UTC
Permalink
Post by Jayachandran C
Post by Rafael J. Wysocki
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi
[cut]
Post by Jayachandran C
If we are moving the ACPI/PCI code from drivers/acpi to
arch/arm64/ , there is an issue in having the header file
ecam.h in drivers/pci
The current include of "../pci/ecam.h" is slightly ugly (Arnd
and David had already noted this), but including the driver
header from arch code would be even worse.
I can either merge ecam.h into include/linux/pci.h
or move it to a new file include/linux/pci-ecam.h, any
suggestion on which is preferable?
My preference would be pci-ecam.h as we did a similar thing for
pci-dma.h, for example, but basically this is up to Bjorn.
Lorenzo Pieralisi
2016-05-13 10:40:03 UTC
Permalink
Post by Rafael J. Wysocki
Post by Jayachandran C
Post by Rafael J. Wysocki
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi
[cut]
Post by Jayachandran C
If we are moving the ACPI/PCI code from drivers/acpi to
arch/arm64/ , there is an issue in having the header file
ecam.h in drivers/pci
The current include of "../pci/ecam.h" is slightly ugly (Arnd
and David had already noted this), but including the driver
header from arch code would be even worse.
I can either merge ecam.h into include/linux/pci.h
or move it to a new file include/linux/pci-ecam.h, any
suggestion on which is preferable?
My preference would be pci-ecam.h as we did a similar thing for
pci-dma.h, for example, but basically this is up to Bjorn.
A word of caution for all interested parties, what we may move
to arch/arm64 (if Catalin and Will are ok with that) here is content
of drivers/acpi/pci_root_generic.c, not drivers/acpi/pci_mcfg.c (and
definitely not the MCFG quirks handling that is coming up next on top
of this series).

I just wanted to make sure we understand that MCFG quirks handling
like eg:

https://lkml.org/lkml/2016/4/28/790

that is coming up following this series has no chance whatsoever to
be handled within arch/arm64, it is just not going to happen.

Maybe I am jumping the gun, I just want to make sure that everyone is
aware that moving part of this series to arch/arm64 has implications,
(and that's why I said that moving part of this code to arch/arm64 is
not as simple as it looks) it may be ok to have an ACPI PCI
implementation that is arch/arm64 specific (mostly for IO space and PCI
resources assignment handling that unfortunately is not uniform across
X86, IA64 and ARM64), but MCFG quirks and related platform code stay out
of arch/arm64 I guess we are all aware of that, just wanted to make
sure :)

Lorenzo
Tomasz Nowicki
2016-05-12 11:00:02 UTC
Permalink
Post by Bjorn Helgaas
Post by Rafael J. Wysocki
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi
Post by Lorenzo Pieralisi
Post by Rafael J. Wysocki
Post by Tomasz Nowicki
This patch provides a way to set the ACPI companion in PCI code.
We define acpi_pci_set_companion() to set the ACPI companion pointer and
call it from PCI core code. The function is stub for now.
---
drivers/pci/probe.c | 2 ++
include/linux/pci-acpi.h | 4 ++++
2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8004f67..fb0b752 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -12,6 +12,7 @@
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/cpumask.h>
+#include <linux/pci-acpi.h>
#include <linux/pci-aspm.h>
#include <linux/aer.h>
#include <linux/acpi.h>
@@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
bridge->dev.parent = parent;
bridge->dev.release = pci_release_host_bridge_dev;
dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
+ acpi_pci_set_companion(bridge);
Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
Post by Tomasz Nowicki
error = pcibios_root_bridge_prepare(bridge);
if (error) {
kfree(bridge);
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 09f9f02..1baa515 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
#endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge)
+{
+}
+
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus)
{
return 0;
--
Honestly, to me it looks like this series is trying very hard to avoid
doing any PCI host bridge configuration stuff from arch/arm64/
although (a) that might be simpler and (b) it would allow us to
identify the code that's common between *all* architectures using ACPI
support for host bridge configuration and to move *that* to a common
place later. As done here it seems to be following the "ARM64 is
generic and the rest of the world is special" line which isn't really
helpful.
I think patch [1-2] should be merged regardless (they may require minor
tweaks if we decide to move pci_acpi_scan_root() to arch/arm64 though,
for include files location). I guess you are referring to patch 8 in
- pci_acpi_scan_root() (and unfortunately all the MCFG/ECAM handling that
goes with it) should live in arch/arm64 or drivers/acpi
To be precise, everything under #ifdef CONFIG_ACPI_PCI_HOST_GENERIC or
equivalent is de facto ARM64-specific, because (as it stands in the
patch series) ARM64 is the only architecture that will select that
option. Unless you are aware of any more architectures planning to
use ACPI (and I'm not aware of any), it will stay the only
architecture selecting it in the foreseeable future.
Therefore you could replace CONFIG_ACPI_PCI_HOST_GENERIC with
CONFIG_ARM64 everywhere in that code which is why in my opinion the
code should live somewhere under arch/arm64/.
Going forward, it should be possible to identify common parts of the
PCI host bridge configuration code in arch/ and move it to
drivers/acpi/ or drivers/pci/, but I bet that won't be the entire code
this series puts under CONFIG_ACPI_PCI_HOST_GENERIC.
The above leads to a quite straightforward conclusion about the order
in which to do things: I'd add ACPI support for PCI host bridge on
ARM64 following what's been done on ia64 (as x86 is more quirky and
kludgy overall) as far as reasonably possible first and then think
about moving common stuff to a common place.
That does seem like a reasonable approach. I had hoped to get more of
this in for v4.7, but we don't have much time left. Maybe some of
Rafael's comments can be addressed by moving and slight restructuring
and we can still squeeze it in.
PCI: Provide common functions for ECAM mapping
PCI: generic, thunder: Use generic ECAM API
PCI, of: Move PCI I/O space management to PCI core code
seem relatively straightforward, and I applied them to pci/arm64 with
the intent of merging them unless there are objections. I made the
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
index 3d52005..e1add01 100644
--- a/drivers/pci/ecam.c
+++ b/drivers/pci/ecam.c
@@ -24,9 +24,9 @@
#include "ecam.h"
/*
- * On 64 bit systems, we do a single ioremap for the whole config space
- * since we have enough virtual address range available. On 32 bit, do an
- * ioremap per bus.
+ * On 64-bit systems, we do a single ioremap for the whole config space
+ * since we have enough virtual address range available. On 32-bit, we
+ * ioremap the config space for each bus individually.
*/
static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);
@@ -42,6 +42,7 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
{
struct pci_config_window *cfg;
unsigned int bus_range, bus_range_max, bsz;
+ struct resource *conflict;
int i, err;
if (busr->start > busr->end)
@@ -58,10 +59,10 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
bus_range = resource_size(&cfg->busr);
bus_range_max = resource_size(cfgres) >> ops->bus_shift;
if (bus_range > bus_range_max) {
- dev_warn(dev, "bus max %#x reduced to %#x",
- bus_range, bus_range_max);
bus_range = bus_range_max;
cfg->busr.end = busr->start + bus_range - 1;
+ dev_warn(dev, "ECAM area %pR can only accommodate %pR (reduced from %pR desired)\n",
+ cfgres, &cfg->busr, busr);
}
bsz = 1 << ops->bus_shift;
@@ -70,9 +71,11 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
cfg->res.flags = IORESOURCE_MEM | IORESOURCE_BUSY;
cfg->res.name = "PCI ECAM";
- err = request_resource(&iomem_resource, &cfg->res);
- if (err) {
- dev_err(dev, "request ECAM res %pR failed\n", &cfg->res);
+ conflict = request_resource(&iomem_resource, &cfg->res);
We need request_resource_conflict here then:
- conflict = request_resource(&iomem_resource, &cfg->res);
+ conflict = request_resource_conflict(&iomem_resource, &cfg->res);

Thanks,
Tomasz
Bjorn Helgaas
2016-05-12 12:10:02 UTC
Permalink
Post by Tomasz Nowicki
Post by Bjorn Helgaas
@@ -70,9 +71,11 @@ struct pci_config_window *pci_ecam_create(struct device *dev,
cfg->res.flags = IORESOURCE_MEM | IORESOURCE_BUSY;
cfg->res.name = "PCI ECAM";
- err = request_resource(&iomem_resource, &cfg->res);
- if (err) {
- dev_err(dev, "request ECAM res %pR failed\n", &cfg->res);
+ conflict = request_resource(&iomem_resource, &cfg->res);
- conflict = request_resource(&iomem_resource, &cfg->res);
+ conflict = request_resource_conflict(&iomem_resource, &cfg->res);
Whoops, fixed, thanks!
Dongdong Liu
2016-05-17 03:20:01 UTC
Permalink
Hi Tomasz

I used the patchset and added "PATCH V6 11/13 specic quirks", tested on HiSilicon D02 board but met the below problem.

[ 2.614115] [<ffffff80083b13bc>] hisi_pcie_init+0x6c/0x1ec
[ 2.619571] [<ffffff80083ab060>] pci_ecam_create+0x130/0x1ec
[ 2.625209] [<ffffff80083f3764>] pci_acpi_scan_root+0x160/0x218
[ 2.631096] [<ffffff80083d1f6c>] acpi_pci_root_add+0x36c/0x42c
[ 2.636897] [<ffffff80083ce36c>] acpi_bus_attach+0xe4/0x1a8
[ 2.642438] [<ffffff80083ce3d8>] acpi_bus_attach+0x150/0x1a8
[ 2.648066] [<ffffff80083ce3d8>] acpi_bus_attach+0x150/0x1a8
[ 2.653693] [<ffffff80083ce55c>] acpi_bus_scan+0x64/0x74
[ 2.658975] [<ffffff8008ae665c>] acpi_scan_init+0x5c/0x19c
[ 2.664429] [<ffffff8008ae6408>] acpi_init+0x280/0x2a4
[ 2.669538] [<ffffff80080829e8>] do_one_initcall+0x8c/0x19c
[ 2.675080] [<ffffff8008ac3af8>] kernel_init_freeable+0x14c/0x1ec
[ 2.681139] [<ffffff80087a8438>] kernel_init+0x10/0xfc
[ 2.686248] [<ffffff8008085e10>] ret_from_fork+0x10/0x40

In hisi_pcie_init, I used "struct acpi_device *device = ACPI_COMPANION(dev);".
I found the reason is V7 lack the below code. I added the below code, it worked ok.

[PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code.
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -564,6 +564,11 @@ static int acpi_pci_root_add(struct acpi_device *device,
}
}

+ /*
+ * pci_create_root_bus() needs to detect the parent device type,
+ * so initialize its companion data accordingly.
+ */
+ ACPI_COMPANION_SET(&device->dev, device);

This code will be upstreamed with the "PATCH V6 11/13 specic quirks" in next time after the patchset is accepted.
Right ?
Post by Tomasz Nowicki
This patch provides a way to set the ACPI companion in PCI code.
We define acpi_pci_set_companion() to set the ACPI companion pointer and
call it from PCI core code. The function is stub for now.
---
drivers/pci/probe.c | 2 ++
include/linux/pci-acpi.h | 4 ++++
2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8004f67..fb0b752 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -12,6 +12,7 @@
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/cpumask.h>
+#include <linux/pci-acpi.h>
#include <linux/pci-aspm.h>
#include <linux/aer.h>
#include <linux/acpi.h>
@@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
bridge->dev.parent = parent;
bridge->dev.release = pci_release_host_bridge_dev;
dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
+ acpi_pci_set_companion(bridge);
error = pcibios_root_bridge_prepare(bridge);
if (error) {
kfree(bridge);
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 09f9f02..1baa515 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
#endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge)
+{
+}
+
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus)
{
return 0;
Tomasz Nowicki
2016-05-17 13:50:01 UTC
Permalink
Post by Gabriele Paoloni
Hi Tomasz
I used the patchset and added "PATCH V6 11/13 specic quirks", tested on
HiSilicon D02 board but met the below problem.
[ 2.614115] [<ffffff80083b13bc>] hisi_pcie_init+0x6c/0x1ec
[ 2.619571] [<ffffff80083ab060>] pci_ecam_create+0x130/0x1ec
[ 2.625209] [<ffffff80083f3764>] pci_acpi_scan_root+0x160/0x218
[ 2.631096] [<ffffff80083d1f6c>] acpi_pci_root_add+0x36c/0x42c
[ 2.636897] [<ffffff80083ce36c>] acpi_bus_attach+0xe4/0x1a8
[ 2.642438] [<ffffff80083ce3d8>] acpi_bus_attach+0x150/0x1a8
[ 2.648066] [<ffffff80083ce3d8>] acpi_bus_attach+0x150/0x1a8
[ 2.653693] [<ffffff80083ce55c>] acpi_bus_scan+0x64/0x74
[ 2.658975] [<ffffff8008ae665c>] acpi_scan_init+0x5c/0x19c
[ 2.664429] [<ffffff8008ae6408>] acpi_init+0x280/0x2a4
[ 2.669538] [<ffffff80080829e8>] do_one_initcall+0x8c/0x19c
[ 2.675080] [<ffffff8008ac3af8>] kernel_init_freeable+0x14c/0x1ec
[ 2.681139] [<ffffff80087a8438>] kernel_init+0x10/0xfc
[ 2.686248] [<ffffff8008085e10>] ret_from_fork+0x10/0x40
In hisi_pcie_init, I used "struct acpi_device *device =
ACPI_COMPANION(dev);".
I found the reason is V7 lack the below code. I added the below code, it worked ok.
[PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device
companion assignment to core code.
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -564,6 +564,11 @@ static int acpi_pci_root_add(struct acpi_device *device,
}
}
+ /*
+ * pci_create_root_bus() needs to detect the parent device type,
+ * so initialize its companion data accordingly.
+ */
+ ACPI_COMPANION_SET(&device->dev, device);
This code will be upstreamed with the "PATCH V6 11/13 specic quirks" in
next time after the patchset is accepted.
Right ?
We had that patch in previous series to retrieve PCI domain nicely. But
that has bad implication to userspace. See:
https://lkml.org/lkml/2016/5/9/918

I understand that:
[PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device
companion assignment to core code.
helps to get firmware specific info in hisi_pcie_init but we need to
figure out something better for quirk handling too.

Tomasz
Tomasz Nowicki
2016-05-10 15:30:02 UTC
Permalink
This patch is going to implement generic PCI host controller for
ACPI world, similar to what pci-host-generic.c driver does for DT world.

All such drivers, which we have seen so far, were implemented within
arch/ directory since they had some arch assumptions (x86 and ia64).
However, they all are doing similar thing, so it makes sense to find
some common code and abstract it into the generic driver.

In order to handle PCI config space regions properly, we define new
MCFG interface which does sanity checks on MCFG table and keeps its
root pointer. User is able to lookup MCFG regions based on that root
pointer and specified domain:bus_start:bus_end touple. We are using
pci_mmcfg_late_init old prototype to avoid another function name.

The implementation of pci_acpi_scan_root() looks up the MCFG entries
and sets up a new mapping (regions are not mapped until host controller ask
for it). Generic PCI functions are used for accessing config space.
Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h
to create and access ECAM mappings.

As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
should be made on a per-architecture basis.

Signed-off-by: Tomasz Nowicki <***@semihalf.com>
Signed-off-by: Jayachandran C <***@broadcom.com>
---
drivers/acpi/Kconfig | 8 +++
drivers/acpi/Makefile | 1 +
drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++++++
drivers/acpi/pci_root_generic.c | 149 ++++++++++++++++++++++++++++++++++++++++
drivers/pci/ecam.h | 5 ++
include/linux/pci-acpi.h | 5 ++
include/linux/pci.h | 5 +-
7 files changed, 269 insertions(+), 1 deletion(-)
create mode 100644 drivers/acpi/pci_mcfg.c
create mode 100644 drivers/acpi/pci_root_generic.c

diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index 183ffa3..44afc76 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -346,6 +346,14 @@ config ACPI_PCI_SLOT
i.e., segment/bus/device/function tuples, with physical slots in
the system. If you are unsure, say N.

+config ACPI_PCI_HOST_GENERIC
+ bool
+ select PCI_ECAM
+ help
+ Select this config option from the architecture Kconfig,
+ if it is preferred to enable ACPI PCI host controller driver which
+ has no arch-specific assumptions.
+
config X86_PM_TIMER
bool "Power Management Timer Support" if EXPERT
depends on X86
diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
index 81e5cbc..627a2b7 100644
--- a/drivers/acpi/Makefile
+++ b/drivers/acpi/Makefile
@@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
acpi-y += ec.o
acpi-$(CONFIG_ACPI_DOCK) += dock.o
acpi-y += pci_root.o pci_link.o pci_irq.o
+obj-$(CONFIG_ACPI_PCI_HOST_GENERIC) += pci_root_generic.o pci_mcfg.o
acpi-y += acpi_lpss.o acpi_apd.o
acpi-y += acpi_platform.o
acpi-y += acpi_pnp.o
diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
new file mode 100644
index 0000000..373d079
--- /dev/null
+++ b/drivers/acpi/pci_mcfg.c
@@ -0,0 +1,97 @@
+/*
+ * Copyright (C) 2016 Broadcom
+ * Author: Jayachandran C <***@broadcom.com>
+ * Copyright (C) 2016 Semihalf
+ * Author: Tomasz Nowicki <***@semihalf.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/pci-acpi.h>
+
+#define PREFIX "ACPI: "
+
+/* Root pointer to the mapped MCFG table */
+static struct acpi_table_mcfg *mcfg_table;
+
+#define MCFG_ENTRIES(mcfg_ptr) (((mcfg_ptr)->header.length - \
+ sizeof(struct acpi_table_mcfg)) / \
+ sizeof(struct acpi_mcfg_allocation))
+
+static phys_addr_t pci_mcfg_lookup_static(u16 seg, u8 bus_start, u8 bus_end)
+{
+ struct acpi_mcfg_allocation *mptr;
+ int i;
+
+ if (!mcfg_table) {
+ pr_err(PREFIX "MCFG table not available, lookup failed\n");
+ return -ENXIO;
+ }
+
+ mptr = (struct acpi_mcfg_allocation *) &mcfg_table[1];
+
+ /*
+ * We expect exact match, unless MCFG entry end bus covers more than
+ * specified by caller.
+ */
+ for (i = 0; i < MCFG_ENTRIES(mcfg_table); i++, mptr++) {
+ if (mptr->pci_segment == seg &&
+ mptr->start_bus_number == bus_start &&
+ mptr->end_bus_number >= bus_end) {
+ return mptr->address;
+ }
+ }
+
+ return -ENXIO;
+}
+
+phys_addr_t pci_mcfg_lookup(struct acpi_device *device, u16 seg,
+ struct resource *bus_res)
+{
+ phys_addr_t addr;
+
+ addr = acpi_pci_root_get_mcfg_addr(device->handle);
+ if (addr)
+ return addr;
+
+ return pci_mcfg_lookup_static(seg, bus_res->start, bus_res->end);
+}
+
+static __init int pci_mcfg_parse(struct acpi_table_header *header)
+{
+ struct acpi_table_mcfg *mcfg;
+ int n;
+
+ if (!header)
+ return -EINVAL;
+
+ mcfg = (struct acpi_table_mcfg *)header;
+ n = MCFG_ENTRIES(mcfg);
+ if (n <= 0 || n > 255) {
+ pr_err(PREFIX "MCFG has incorrect entries (%d).\n", n);
+ return -EINVAL;
+ }
+
+ mcfg_table = mcfg;
+ pr_info(PREFIX "MCFG table loaded, %d entries detected\n", n);
+ return 0;
+}
+
+/* Interface called by ACPI - parse and save MCFG table */
+void __init pci_mmcfg_late_init(void)
+{
+ int err = acpi_table_parse(ACPI_SIG_MCFG, pci_mcfg_parse);
+ if (err)
+ pr_err(PREFIX "Failed to parse MCFG (%d)\n", err);
+}
diff --git a/drivers/acpi/pci_root_generic.c b/drivers/acpi/pci_root_generic.c
new file mode 100644
index 0000000..6f4940a
--- /dev/null
+++ b/drivers/acpi/pci_root_generic.c
@@ -0,0 +1,149 @@
+/*
+ * Copyright (C) 2016 Broadcom
+ * Author: Jayachandran C <***@broadcom.com>
+ * Copyright (C) 2016 Semihalf
+ * Author: Tomasz Nowicki <***@semihalf.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/pci-acpi.h>
+#include <linux/slab.h>
+
+#include "../pci/ecam.h"
+
+#define PREFIX "ACPI PCI: "
+
+/* ACPI info for generic ACPI PCI controller */
+struct acpi_pci_generic_root_info {
+ struct acpi_pci_root_info common;
+ struct pci_config_window *cfg; /* config space mapping */
+};
+
+void acpi_pci_set_companion(struct pci_host_bridge *bridge)
+{
+ struct pci_config_window *cfg = bridge->bus->sysdata;
+
+ ACPI_COMPANION_SET(&bridge->dev, cfg->companion);
+}
+
+int acpi_pci_bus_domain_nr(struct pci_bus *bus)
+{
+ struct pci_config_window *cfg = bus->sysdata;
+
+ return cfg->domain;
+}
+
+/*
+ * Lookup the bus range for the domain in MCFG, and set up config space
+ * mapping.
+ */
+static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root,
+ struct acpi_pci_generic_root_info *ri)
+{
+ struct resource *bus_res = &root->secondary;
+ u16 seg = root->segment;
+ struct pci_config_window *cfg;
+ struct resource cfgres;
+ unsigned int bsz;
+ phys_addr_t addr;
+
+ addr = pci_mcfg_lookup(root->device, seg, bus_res);
+ if (IS_ERR_VALUE(addr)) {
+ pr_err(PREFIX"%04x:%pR MCFG region not found\n", seg, bus_res);
+ return addr;
+ }
+
+ bsz = 1 << pci_generic_ecam_ops.bus_shift;
+ cfgres.start = addr + bus_res->start * bsz;
+ cfgres.end = addr + (bus_res->end + 1) * bsz - 1;
+ cfgres.flags = IORESOURCE_MEM;
+ cfg = pci_ecam_create(&root->device->dev, &cfgres, bus_res,
+ &pci_generic_ecam_ops);
+ if (IS_ERR(cfg)) {
+ pr_err("%04x:%pR error %ld mapping CAM\n", seg, bus_res,
+ PTR_ERR(cfg));
+ return PTR_ERR(cfg);
+ }
+
+ cfg->domain = seg;
+ cfg->companion = root->device;
+ ri->cfg = cfg;
+ return 0;
+}
+
+/* release_info: free resrouces allocated by init_info */
+static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci)
+{
+ struct acpi_pci_generic_root_info *ri;
+
+ ri = container_of(ci, struct acpi_pci_generic_root_info, common);
+ pci_ecam_free(ri->cfg);
+ kfree(ri);
+}
+
+static struct acpi_pci_root_ops acpi_pci_root_ops = {
+ .release_info = pci_acpi_generic_release_info,
+};
+
+/* Interface called from ACPI code to setup PCI host controller */
+struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
+{
+ int node = acpi_get_node(root->device->handle);
+ struct acpi_pci_generic_root_info *ri;
+ struct pci_bus *bus, *child;
+ int err;
+
+ ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node);
+ if (!ri)
+ return NULL;
+
+ err = pci_acpi_setup_ecam_mapping(root, ri);
+ if (err)
+ return NULL;
+
+ acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops;
+ bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common,
+ ri->cfg);
+ if (!bus)
+ return NULL;
+
+ pci_bus_size_bridges(bus);
+ pci_bus_assign_resources(bus);
+
+ list_for_each_entry(child, &bus->children, node)
+ pcie_bus_configure_settings(child);
+
+ return bus;
+}
+
+int raw_pci_read(unsigned int domain, unsigned int busn, unsigned int devfn,
+ int reg, int len, u32 *val)
+{
+ struct pci_bus *bus = pci_find_bus(domain, busn);
+
+ if (!bus)
+ return PCIBIOS_DEVICE_NOT_FOUND;
+ return bus->ops->read(bus, devfn, reg, len, val);
+}
+
+int raw_pci_write(unsigned int domain, unsigned int busn, unsigned int devfn,
+ int reg, int len, u32 val)
+{
+ struct pci_bus *bus = pci_find_bus(domain, busn);
+
+ if (!bus)
+ return PCIBIOS_DEVICE_NOT_FOUND;
+ return bus->ops->write(bus, devfn, reg, len, val);
+}
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
index 1ad2176..1cccf57 100644
--- a/drivers/pci/ecam.h
+++ b/drivers/pci/ecam.h
@@ -45,6 +45,11 @@ struct pci_config_window {
void __iomem *win; /* 64-bit single mapping */
void __iomem **winp; /* 32-bit per bus mapping */
};
+#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
+ struct acpi_device *companion; /* ACPI companion device */
+#endif
+ int domain;
+
};

/* create and free for pci_config_window */
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 1baa515..42ff844 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
#endif /* CONFIG_ACPI */

+#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
+void acpi_pci_set_companion(struct pci_host_bridge *bridge);
+int acpi_pci_bus_domain_nr(struct pci_bus *bus);
+#else
static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge)
{
}
@@ -119,6 +123,7 @@ static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus)
{
return 0;
}
+#endif

#ifdef CONFIG_ACPI_APEI
extern bool aer_acpi_firmware_first(void);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index d6ea6ce..b2e8886 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1722,7 +1722,10 @@ void pcibios_free_irq(struct pci_dev *dev);
extern struct dev_pm_ops pcibios_pm_ops;
#endif

-#ifdef CONFIG_PCI_MMCONFIG
+#if defined(CONFIG_PCI_MMCONFIG) || defined(CONFIG_ACPI_PCI_HOST_GENERIC)
+struct acpi_device;
+phys_addr_t pci_mcfg_lookup(struct acpi_device *device, u16 seg,
+ struct resource *bus_res);
void __init pci_mmcfg_early_init(void);
void __init pci_mmcfg_late_init(void);
#else
--
1.9.1
Rafael J. Wysocki
2016-05-10 18:00:03 UTC
Permalink
Post by Tomasz Nowicki
This patch is going to implement generic PCI host controller for
ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within
arch/ directory since they had some arch assumptions (x86 and ia64).
However, they all are doing similar thing, so it makes sense to find
some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new
MCFG interface which does sanity checks on MCFG table and keeps its
root pointer. User is able to lookup MCFG regions based on that root
pointer and specified domain:bus_start:bus_end touple. We are using
pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries
and sets up a new mapping (regions are not mapped until host controller ask
for it). Generic PCI functions are used for accessing config space.
Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h
to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
should be made on a per-architecture basis.
---
drivers/acpi/Kconfig | 8 +++
drivers/acpi/Makefile | 1 +
drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++++++
drivers/acpi/pci_root_generic.c | 149 ++++++++++++++++++++++++++++++++++++++++
Why do we need a new file? Would there be any problem with adding
that code to pci_root.c?
Rafael J. Wysocki
2016-05-10 18:20:02 UTC
Permalink
Post by Tomasz Nowicki
This patch is going to implement generic PCI host controller for
ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within
arch/ directory since they had some arch assumptions (x86 and ia64).
However, they all are doing similar thing, so it makes sense to find
some common code and abstract it into the generic driver.
Does it mean x86 and ia64 will now be able to use this code too?
Post by Tomasz Nowicki
In order to handle PCI config space regions properly, we define new
MCFG interface which does sanity checks on MCFG table and keeps its
root pointer. User is able to lookup MCFG regions based on that root
pointer and specified domain:bus_start:bus_end touple. We are using
pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries
and sets up a new mapping (regions are not mapped until host controller ask
for it). Generic PCI functions are used for accessing config space.
Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h
to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
should be made on a per-architecture basis.
If that code really is generic and there will be more than one
architecture using it ever, I think it'll be better for the
architectures that don't use it to set something like
ARCH_ACPI_PCI_HOST and whoever doesn't set that will use the generic
thing. That'd be more logical at least IMO.
Post by Tomasz Nowicki
---
drivers/acpi/Kconfig | 8 +++
drivers/acpi/Makefile | 1 +
drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++++++
drivers/acpi/pci_root_generic.c | 149 ++++++++++++++++++++++++++++++++++++++++
drivers/pci/ecam.h | 5 ++
include/linux/pci-acpi.h | 5 ++
include/linux/pci.h | 5 +-
7 files changed, 269 insertions(+), 1 deletion(-)
create mode 100644 drivers/acpi/pci_mcfg.c
create mode 100644 drivers/acpi/pci_root_generic.c
diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index 183ffa3..44afc76 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -346,6 +346,14 @@ config ACPI_PCI_SLOT
i.e., segment/bus/device/function tuples, with physical slots in
the system. If you are unsure, say N.
+config ACPI_PCI_HOST_GENERIC
+ bool
+ select PCI_ECAM
+ help
+ Select this config option from the architecture Kconfig,
+ if it is preferred to enable ACPI PCI host controller driver which
+ has no arch-specific assumptions.
+
config X86_PM_TIMER
bool "Power Management Timer Support" if EXPERT
depends on X86
diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
index 81e5cbc..627a2b7 100644
--- a/drivers/acpi/Makefile
+++ b/drivers/acpi/Makefile
@@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
acpi-y += ec.o
acpi-$(CONFIG_ACPI_DOCK) += dock.o
acpi-y += pci_root.o pci_link.o pci_irq.o
+obj-$(CONFIG_ACPI_PCI_HOST_GENERIC) += pci_root_generic.o pci_mcfg.o
acpi-y += acpi_lpss.o acpi_apd.o
acpi-y += acpi_platform.o
acpi-y += acpi_pnp.o
diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
new file mode 100644
index 0000000..373d079
--- /dev/null
+++ b/drivers/acpi/pci_mcfg.c
@@ -0,0 +1,97 @@
+/*
+ * Copyright (C) 2016 Broadcom
+ * Copyright (C) 2016 Semihalf
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/pci-acpi.h>
+
+#define PREFIX "ACPI: "
If that is a new file (and I'm totally unconvinced about the need for
it), can we simply define a pr_fmt() here as all messages in it seem
to be printed by the pr_* functions?
Jayachandran C
2016-05-13 11:30:01 UTC
Permalink
Post by Tomasz Nowicki
This patch is going to implement generic PCI host controller for
ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within
arch/ directory since they had some arch assumptions (x86 and ia64).
However, they all are doing similar thing, so it makes sense to find
some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new
MCFG interface which does sanity checks on MCFG table and keeps its
root pointer. User is able to lookup MCFG regions based on that root
pointer and specified domain:bus_start:bus_end touple. We are using
pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries
and sets up a new mapping (regions are not mapped until host controller ask
for it). Generic PCI functions are used for accessing config space.
Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h
to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
should be made on a per-architecture basis.
---
[....]
Post by Tomasz Nowicki
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
index 1ad2176..1cccf57 100644
--- a/drivers/pci/ecam.h
+++ b/drivers/pci/ecam.h
@@ -45,6 +45,11 @@ struct pci_config_window {
void __iomem *win; /* 64-bit single mapping */
void __iomem **winp; /* 32-bit per bus mapping */
};
+#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
+ struct acpi_device *companion; /* ACPI companion device */
+#endif
+ int domain;
+
};
Using struct pci_config_window to pass along domain and
companion looks bad. I think there are two possible options
to do this better:

1. add a 'struct fwnode_handle *' or 'struct device *parent_dev'
instead of the companion and domain fields above. In case of
ACPI either of them can be used to get the acpi_device and
both domain and companion can be set from that.

2. make pci_config_window fully embeddable by moving allocation
out of pci_ecam_create to its callers. Then it can be embedded
into acpi_pci_generic_root_info, and container_of can be used
to get acpi info from ->sysdata.

The first option should be easier to implement but the second may
be better on long run. I would leave it to the Bjorn or Rafael to
suggest which is preferred.

JC.
Rafael J. Wysocki
2016-05-13 11:40:01 UTC
Permalink
Post by Jayachandran C
Post by Tomasz Nowicki
This patch is going to implement generic PCI host controller for
ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within
arch/ directory since they had some arch assumptions (x86 and ia64).
However, they all are doing similar thing, so it makes sense to find
some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new
MCFG interface which does sanity checks on MCFG table and keeps its
root pointer. User is able to lookup MCFG regions based on that root
pointer and specified domain:bus_start:bus_end touple. We are using
pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries
and sets up a new mapping (regions are not mapped until host controller ask
for it). Generic PCI functions are used for accessing config space.
Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h
to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
should be made on a per-architecture basis.
---
[....]
Post by Tomasz Nowicki
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
index 1ad2176..1cccf57 100644
--- a/drivers/pci/ecam.h
+++ b/drivers/pci/ecam.h
@@ -45,6 +45,11 @@ struct pci_config_window {
void __iomem *win; /* 64-bit single mapping */
void __iomem **winp; /* 32-bit per bus mapping */
};
+#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
+ struct acpi_device *companion; /* ACPI companion device */
+#endif
+ int domain;
+
};
Using struct pci_config_window to pass along domain and
companion looks bad. I think there are two possible options
1. add a 'struct fwnode_handle *' or 'struct device *parent_dev'
instead of the companion and domain fields above. In case of
ACPI either of them can be used to get the acpi_device and
both domain and companion can be set from that.
2. make pci_config_window fully embeddable by moving allocation
out of pci_ecam_create to its callers. Then it can be embedded
into acpi_pci_generic_root_info, and container_of can be used
to get acpi info from ->sysdata.
The first option should be easier to implement but the second may
be better on long run. I would leave it to the Bjorn or Rafael to
suggest which is preferred.
Personally, I'd probably try to use fwnode_handle, but the second
option makes sense too in principle.
Tomasz Nowicki
2016-05-13 11:50:01 UTC
Permalink
Post by Rafael J. Wysocki
Post by Jayachandran C
Post by Tomasz Nowicki
This patch is going to implement generic PCI host controller for
ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within
arch/ directory since they had some arch assumptions (x86 and ia64).
However, they all are doing similar thing, so it makes sense to find
some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new
MCFG interface which does sanity checks on MCFG table and keeps its
root pointer. User is able to lookup MCFG regions based on that root
pointer and specified domain:bus_start:bus_end touple. We are using
pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries
and sets up a new mapping (regions are not mapped until host controller ask
for it). Generic PCI functions are used for accessing config space.
Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h
to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
should be made on a per-architecture basis.
---
[....]
Post by Tomasz Nowicki
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
index 1ad2176..1cccf57 100644
--- a/drivers/pci/ecam.h
+++ b/drivers/pci/ecam.h
@@ -45,6 +45,11 @@ struct pci_config_window {
void __iomem *win; /* 64-bit single mapping */
void __iomem **winp; /* 32-bit per bus mapping */
};
+#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
+ struct acpi_device *companion; /* ACPI companion device */
+#endif
+ int domain;
+
};
Using struct pci_config_window to pass along domain and
companion looks bad. I think there are two possible options
1. add a 'struct fwnode_handle *' or 'struct device *parent_dev'
instead of the companion and domain fields above. In case of
ACPI either of them can be used to get the acpi_device and
both domain and companion can be set from that.
2. make pci_config_window fully embeddable by moving allocation
out of pci_ecam_create to its callers. Then it can be embedded
into acpi_pci_generic_root_info, and container_of can be used
to get acpi info from ->sysdata.
The first option should be easier to implement but the second may
be better on long run. I would leave it to the Bjorn or Rafael to
suggest which is preferred.
Personally, I'd probably try to use fwnode_handle, but the second
option makes sense too in principle.
Thanks for suggestions. I will try to use fwnode_handle

Tomasz
Jayachandran C
2016-05-14 09:10:01 UTC
Permalink
Post by Tomasz Nowicki
This patch is going to implement generic PCI host controller for
ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within
arch/ directory since they had some arch assumptions (x86 and ia64).
However, they all are doing similar thing, so it makes sense to find
some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new
MCFG interface which does sanity checks on MCFG table and keeps its
root pointer. User is able to lookup MCFG regions based on that root
pointer and specified domain:bus_start:bus_end touple. We are using
pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries
and sets up a new mapping (regions are not mapped until host controller ask
for it). Generic PCI functions are used for accessing config space.
Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h
to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
should be made on a per-architecture basis.
Looking thru the new code, I see a few issues, please see below
Post by Tomasz Nowicki
---
drivers/acpi/Kconfig | 8 +++
drivers/acpi/Makefile | 1 +
drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++++++
drivers/acpi/pci_root_generic.c | 149 ++++++++++++++++++++++++++++++++++++++++
drivers/pci/ecam.h | 5 ++
include/linux/pci-acpi.h | 5 ++
include/linux/pci.h | 5 +-
7 files changed, 269 insertions(+), 1 deletion(-)
create mode 100644 drivers/acpi/pci_mcfg.c
create mode 100644 drivers/acpi/pci_root_generic.c
diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index 183ffa3..44afc76 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -346,6 +346,14 @@ config ACPI_PCI_SLOT
i.e., segment/bus/device/function tuples, with physical slots in
the system. If you are unsure, say N.
+config ACPI_PCI_HOST_GENERIC
+ bool
+ select PCI_ECAM
+ help
+ Select this config option from the architecture Kconfig,
+ if it is preferred to enable ACPI PCI host controller driver which
+ has no arch-specific assumptions.
+
config X86_PM_TIMER
bool "Power Management Timer Support" if EXPERT
depends on X86
diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
index 81e5cbc..627a2b7 100644
--- a/drivers/acpi/Makefile
+++ b/drivers/acpi/Makefile
@@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
acpi-y += ec.o
acpi-$(CONFIG_ACPI_DOCK) += dock.o
acpi-y += pci_root.o pci_link.o pci_irq.o
+obj-$(CONFIG_ACPI_PCI_HOST_GENERIC) += pci_root_generic.o pci_mcfg.o
acpi-y += acpi_lpss.o acpi_apd.o
acpi-y += acpi_platform.o
acpi-y += acpi_pnp.o
diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
new file mode 100644
index 0000000..373d079
--- /dev/null
+++ b/drivers/acpi/pci_mcfg.c
@@ -0,0 +1,97 @@
+/*
+ * Copyright (C) 2016 Broadcom
+ * Copyright (C) 2016 Semihalf
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/pci-acpi.h>
+
+#define PREFIX "ACPI: "
+
+/* Root pointer to the mapped MCFG table */
+static struct acpi_table_mcfg *mcfg_table;
+
+#define MCFG_ENTRIES(mcfg_ptr) (((mcfg_ptr)->header.length - \
+ sizeof(struct acpi_table_mcfg)) / \
+ sizeof(struct acpi_mcfg_allocation))
It would be better if you used static inline function here.
Post by Tomasz Nowicki
+static phys_addr_t pci_mcfg_lookup_static(u16 seg, u8 bus_start, u8 bus_end)
+{
+ struct acpi_mcfg_allocation *mptr;
+ int i;
+
+ if (!mcfg_table) {
+ pr_err(PREFIX "MCFG table not available, lookup failed\n");
+ return -ENXIO;
+ }
+
+ mptr = (struct acpi_mcfg_allocation *) &mcfg_table[1];
+
+ /*
+ * We expect exact match, unless MCFG entry end bus covers more than
+ * specified by caller.
+ */
+ for (i = 0; i < MCFG_ENTRIES(mcfg_table); i++, mptr++) {
+ if (mptr->pci_segment == seg &&
+ mptr->start_bus_number == bus_start &&
+ mptr->end_bus_number >= bus_end) {
+ return mptr->address;
+ }
+ }
There is an issue here, the bus range is obtained if different
ways. If the _CRS has it, this should be fine. But if it is _BBN-0xff
or the default 0-0xff, then I would think taking the MCFG entry end
would be better. This would require updating the bus resource end.

Also (trivial), the braces are not needed.
Post by Tomasz Nowicki
+ return -ENXIO;
+}
+
+phys_addr_t pci_mcfg_lookup(struct acpi_device *device, u16 seg,
+ struct resource *bus_res)
+{
+ phys_addr_t addr;
+
+ addr = acpi_pci_root_get_mcfg_addr(device->handle);
+ if (addr)
+ return addr;
When you have address from _CBA, you are assuming that the
bus range is also set correctly (from _CRS or as _BBN-0xff). Is
this assumption correct?

(this was discussed earlier) you are doing the _CBA call again,
I think cleaning up the _CBA, _BBN, _CRS and MCFG based
ECAM area resources and defaults should be a different patch,
which would need changes to pci_root.c. We should use
root->mcfg_addr in this patchset.
Post by Tomasz Nowicki
+
+ return pci_mcfg_lookup_static(seg, bus_res->start, bus_res->end);
There is no need to have a separate function for this, based on by
above comment, you might need to change bus_res, so having it here
would be better.
Post by Tomasz Nowicki
+}
+
+static __init int pci_mcfg_parse(struct acpi_table_header *header)
+{
+ struct acpi_table_mcfg *mcfg;
+ int n;
+
+ if (!header)
+ return -EINVAL;
This is not needed, the handler is not called if header is NULL
Post by Tomasz Nowicki
+
+ mcfg = (struct acpi_table_mcfg *)header;
+ n = MCFG_ENTRIES(mcfg);
+ if (n <= 0 || n > 255) {
+ pr_err(PREFIX "MCFG has incorrect entries (%d).\n", n);
+ return -EINVAL;
+ }
+ mcfg_table = mcfg;
Saving a reference of ACPI mapping seems dangerous, acpi_parse_table
calles early_acpi_os_unmap_memory() after calling handler, which does
not do anything since acpi_gbl_permanent_mmap is set.

I would suggest copying the entries.
Post by Tomasz Nowicki
+ pr_info(PREFIX "MCFG table loaded, %d entries detected\n", n);
+ return 0;
+}
+
+/* Interface called by ACPI - parse and save MCFG table */
+void __init pci_mmcfg_late_init(void)
+{
+ int err = acpi_table_parse(ACPI_SIG_MCFG, pci_mcfg_parse);
+ if (err)
+ pr_err(PREFIX "Failed to parse MCFG (%d)\n", err);
+}
JC.
Tomasz Nowicki
2016-05-23 11:40:01 UTC
Permalink
Post by Jayachandran C
Post by Tomasz Nowicki
This patch is going to implement generic PCI host controller for
ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within
arch/ directory since they had some arch assumptions (x86 and ia64).
However, they all are doing similar thing, so it makes sense to find
some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new
MCFG interface which does sanity checks on MCFG table and keeps its
root pointer. User is able to lookup MCFG regions based on that root
pointer and specified domain:bus_start:bus_end touple. We are using
pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries
and sets up a new mapping (regions are not mapped until host controller ask
for it). Generic PCI functions are used for accessing config space.
Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h
to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
should be made on a per-architecture basis.
Looking thru the new code, I see a few issues, please see below
Post by Tomasz Nowicki
---
drivers/acpi/Kconfig | 8 +++
drivers/acpi/Makefile | 1 +
drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++++++
drivers/acpi/pci_root_generic.c | 149 ++++++++++++++++++++++++++++++++++++++++
drivers/pci/ecam.h | 5 ++
include/linux/pci-acpi.h | 5 ++
include/linux/pci.h | 5 +-
7 files changed, 269 insertions(+), 1 deletion(-)
create mode 100644 drivers/acpi/pci_mcfg.c
create mode 100644 drivers/acpi/pci_root_generic.c
diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index 183ffa3..44afc76 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -346,6 +346,14 @@ config ACPI_PCI_SLOT
i.e., segment/bus/device/function tuples, with physical slots in
the system. If you are unsure, say N.
+config ACPI_PCI_HOST_GENERIC
+ bool
+ select PCI_ECAM
+ help
+ Select this config option from the architecture Kconfig,
+ if it is preferred to enable ACPI PCI host controller driver which
+ has no arch-specific assumptions.
+
config X86_PM_TIMER
bool "Power Management Timer Support" if EXPERT
depends on X86
diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
index 81e5cbc..627a2b7 100644
--- a/drivers/acpi/Makefile
+++ b/drivers/acpi/Makefile
@@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
acpi-y += ec.o
acpi-$(CONFIG_ACPI_DOCK) += dock.o
acpi-y += pci_root.o pci_link.o pci_irq.o
+obj-$(CONFIG_ACPI_PCI_HOST_GENERIC) += pci_root_generic.o pci_mcfg.o
acpi-y += acpi_lpss.o acpi_apd.o
acpi-y += acpi_platform.o
acpi-y += acpi_pnp.o
diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
new file mode 100644
index 0000000..373d079
--- /dev/null
+++ b/drivers/acpi/pci_mcfg.c
@@ -0,0 +1,97 @@
+/*
+ * Copyright (C) 2016 Broadcom
+ * Copyright (C) 2016 Semihalf
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/pci-acpi.h>
+
+#define PREFIX "ACPI: "
+
+/* Root pointer to the mapped MCFG table */
+static struct acpi_table_mcfg *mcfg_table;
+
+#define MCFG_ENTRIES(mcfg_ptr) (((mcfg_ptr)->header.length - \
+ sizeof(struct acpi_table_mcfg)) / \
+ sizeof(struct acpi_mcfg_allocation))
It would be better if you used static inline function here.
OK
Post by Jayachandran C
Post by Tomasz Nowicki
+static phys_addr_t pci_mcfg_lookup_static(u16 seg, u8 bus_start, u8 bus_end)
+{
+ struct acpi_mcfg_allocation *mptr;
+ int i;
+
+ if (!mcfg_table) {
+ pr_err(PREFIX "MCFG table not available, lookup failed\n");
+ return -ENXIO;
+ }
+
+ mptr = (struct acpi_mcfg_allocation *) &mcfg_table[1];
+
+ /*
+ * We expect exact match, unless MCFG entry end bus covers more than
+ * specified by caller.
+ */
+ for (i = 0; i < MCFG_ENTRIES(mcfg_table); i++, mptr++) {
+ if (mptr->pci_segment == seg &&
+ mptr->start_bus_number == bus_start &&
+ mptr->end_bus_number >= bus_end) {
+ return mptr->address;
+ }
+ }
There is an issue here, the bus range is obtained if different
ways. If the _CRS has it, this should be fine. But if it is _BBN-0xff
or the default 0-0xff, then I would think taking the MCFG entry end
would be better. This would require updating the bus resource end.
Yeah we can implement algorithm like that but IMO it would try to guess
what FW designer meant putting various combination of BUS resources into
the tables. We should rather be strict about this so I agree with Bjorn:
https://lkml.org/lkml/2016/4/28/807
Either we get exact match or MCFG covers more than the host bridge range.

Host bridge bus range definition -> MCFG
_BBN-0xff -> we expect exact _BBN-0xff match
0-0xff -> ditto
_CRS (the only way to trim host bridge bus rage) -> exact match or MCFG
covers more
Post by Jayachandran C
Also (trivial), the braces are not needed.
OK
Post by Jayachandran C
Post by Tomasz Nowicki
+ return -ENXIO;
+}
+
+phys_addr_t pci_mcfg_lookup(struct acpi_device *device, u16 seg,
+ struct resource *bus_res)
+{
+ phys_addr_t addr;
+
+ addr = acpi_pci_root_get_mcfg_addr(device->handle);
+ if (addr)
+ return addr;
When you have address from _CBA, you are assuming that the
bus range is also set correctly (from _CRS or as _BBN-0xff). Is
this assumption correct?
I think so, see above. Maybe Bjorn may advise here.
Post by Jayachandran C
(this was discussed earlier) you are doing the _CBA call again,
I think cleaning up the _CBA, _BBN, _CRS and MCFG based
ECAM area resources and defaults should be a different patch,
which would need changes to pci_root.c. We should use
root->mcfg_addr in this patchset.
OK
Post by Jayachandran C
Post by Tomasz Nowicki
+
+ return pci_mcfg_lookup_static(seg, bus_res->start, bus_res->end);
There is no need to have a separate function for this, based on by
above comment, you might need to change bus_res, so having it here
would be better.
Post by Tomasz Nowicki
+}
+
+static __init int pci_mcfg_parse(struct acpi_table_header *header)
+{
+ struct acpi_table_mcfg *mcfg;
+ int n;
+
+ if (!header)
+ return -EINVAL;
This is not needed, the handler is not called if header is NULL
OK
Post by Jayachandran C
Post by Tomasz Nowicki
+
+ mcfg = (struct acpi_table_mcfg *)header;
+ n = MCFG_ENTRIES(mcfg);
+ if (n <= 0 || n > 255) {
+ pr_err(PREFIX "MCFG has incorrect entries (%d).\n", n);
+ return -EINVAL;
+ }
+ mcfg_table = mcfg;
Saving a reference of ACPI mapping seems dangerous, acpi_parse_table
calles early_acpi_os_unmap_memory() after calling handler, which does
not do anything since acpi_gbl_permanent_mmap is set.
I would suggest copying the entries.
You got the point about early_acpi_os_unmap_memory but that would be the
case if we call pci_mmcfg_late_init before acpi_early_init. But
pci_mmcfg_late_init is called much later. Also the code is much simpler
as is now.

Thanks,
Tomasz
Matthias Brugger
2016-05-19 17:00:01 UTC
Permalink
Post by Tomasz Nowicki
This patch is going to implement generic PCI host controller for
ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within
arch/ directory since they had some arch assumptions (x86 and ia64).
However, they all are doing similar thing, so it makes sense to find
some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new
MCFG interface which does sanity checks on MCFG table and keeps its
root pointer. User is able to lookup MCFG regions based on that root
pointer and specified domain:bus_start:bus_end touple. We are using
pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries
and sets up a new mapping (regions are not mapped until host controller ask
for it). Generic PCI functions are used for accessing config space.
Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h
to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice
should be made on a per-architecture basis.
---
drivers/acpi/Kconfig | 8 +++
drivers/acpi/Makefile | 1 +
drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++++++
drivers/acpi/pci_root_generic.c | 149 ++++++++++++++++++++++++++++++++++++++++
drivers/pci/ecam.h | 5 ++
include/linux/pci-acpi.h | 5 ++
include/linux/pci.h | 5 +-
7 files changed, 269 insertions(+), 1 deletion(-)
create mode 100644 drivers/acpi/pci_mcfg.c
create mode 100644 drivers/acpi/pci_root_generic.c
diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig
index 183ffa3..44afc76 100644
--- a/drivers/acpi/Kconfig
+++ b/drivers/acpi/Kconfig
@@ -346,6 +346,14 @@ config ACPI_PCI_SLOT
i.e., segment/bus/device/function tuples, with physical slots in
the system. If you are unsure, say N.
+config ACPI_PCI_HOST_GENERIC
+ bool
+ select PCI_ECAM
+ help
+ Select this config option from the architecture Kconfig,
+ if it is preferred to enable ACPI PCI host controller driver which
+ has no arch-specific assumptions.
+
config X86_PM_TIMER
bool "Power Management Timer Support" if EXPERT
depends on X86
diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
index 81e5cbc..627a2b7 100644
--- a/drivers/acpi/Makefile
+++ b/drivers/acpi/Makefile
@@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o
acpi-y += ec.o
acpi-$(CONFIG_ACPI_DOCK) += dock.o
acpi-y += pci_root.o pci_link.o pci_irq.o
+obj-$(CONFIG_ACPI_PCI_HOST_GENERIC) += pci_root_generic.o pci_mcfg.o
acpi-y += acpi_lpss.o acpi_apd.o
acpi-y += acpi_platform.o
acpi-y += acpi_pnp.o
diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
new file mode 100644
index 0000000..373d079
--- /dev/null
+++ b/drivers/acpi/pci_mcfg.c
@@ -0,0 +1,97 @@
+/*
+ * Copyright (C) 2016 Broadcom
+ * Copyright (C) 2016 Semihalf
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/pci-acpi.h>
+
+#define PREFIX "ACPI: "
+
+/* Root pointer to the mapped MCFG table */
+static struct acpi_table_mcfg *mcfg_table;
+
+#define MCFG_ENTRIES(mcfg_ptr) (((mcfg_ptr)->header.length - \
+ sizeof(struct acpi_table_mcfg)) / \
+ sizeof(struct acpi_mcfg_allocation))
+
+static phys_addr_t pci_mcfg_lookup_static(u16 seg, u8 bus_start, u8 bus_end)
+{
+ struct acpi_mcfg_allocation *mptr;
+ int i;
+
+ if (!mcfg_table) {
+ pr_err(PREFIX "MCFG table not available, lookup failed\n");
+ return -ENXIO;
+ }
+
+ mptr = (struct acpi_mcfg_allocation *) &mcfg_table[1];
+
+ /*
+ * We expect exact match, unless MCFG entry end bus covers more than
+ * specified by caller.
+ */
+ for (i = 0; i < MCFG_ENTRIES(mcfg_table); i++, mptr++) {
+ if (mptr->pci_segment == seg &&
+ mptr->start_bus_number == bus_start &&
+ mptr->end_bus_number >= bus_end) {
+ return mptr->address;
+ }
+ }
+
+ return -ENXIO;
+}
+
+phys_addr_t pci_mcfg_lookup(struct acpi_device *device, u16 seg,
+ struct resource *bus_res)
+{
+ phys_addr_t addr;
+
+ addr = acpi_pci_root_get_mcfg_addr(device->handle);
+ if (addr)
+ return addr;
+
+ return pci_mcfg_lookup_static(seg, bus_res->start, bus_res->end);
+}
+
+static __init int pci_mcfg_parse(struct acpi_table_header *header)
+{
+ struct acpi_table_mcfg *mcfg;
+ int n;
+
+ if (!header)
+ return -EINVAL;
Maybe that's bike shedding, but acpi_table_parse already checks that the
table passed to the handler exists, so this is redundant.

Regards,
Matthias
Tomasz Nowicki
2016-05-10 15:30:03 UTC
Permalink
Platforms that have memory mapped IO port (such as ARM64) need special
handling for PCI I/O resources. For host bridge's resource probing case
these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.

The same I/O resources need to be released after hotplug
removal so that it can be re-added back by the pci_remap_iospace
function during insertion. As a consequence we unmap I/O resources
with pci_unmap_iospace when we release host bridge resources.

Signed-off-by: Jayachandran C <***@broadcom.com>
Signed-off-by: Sinan Kaya <***@codeaurora.org>
Signed-off-by: Tomasz Nowicki <***@semihalf.com>
---
drivers/acpi/pci_root.c | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)

diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index ae3fe4e..cb3071d 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -719,6 +719,34 @@ next:
resource_list_add_tail(entry, resources);
}
}
+static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
+{
+#ifdef PCI_IOBASE
+ struct resource *res = entry->res;
+ resource_size_t cpu_addr = res->start;
+ resource_size_t pci_addr = cpu_addr - entry->offset;
+ resource_size_t length = resource_size(res);
+ unsigned long port;
+
+ if (pci_register_io_range(cpu_addr, length))
+ goto err;
+
+ port = pci_address_to_pio(cpu_addr);
+ if (port == (unsigned long)-1)
+ goto err;
+
+ res->start = port;
+ res->end = port + length - 1;
+ entry->offset = port - pci_addr;
+
+ if (pci_remap_iospace(res, cpu_addr) < 0)
+ goto err;
+ pr_info("Remapped I/O %pa to %pR\n", &cpu_addr, res);
+ return;
+err:
+ res->flags |= IORESOURCE_DISABLED;
+#endif
+}

int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
{
@@ -740,6 +768,9 @@ int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
"no IO and memory resources present in _CRS\n");
else {
resource_list_for_each_entry_safe(entry, tmp, list) {
+ if (entry->res->flags & IORESOURCE_IO)
+ acpi_pci_root_remap_iospace(entry);
+
if (entry->res->flags & IORESOURCE_DISABLED)
resource_list_destroy_entry(entry);
else
@@ -811,6 +842,8 @@ static void acpi_pci_root_release_info(struct pci_host_bridge *bridge)

resource_list_for_each_entry(entry, &bridge->windows) {
res = entry->res;
+ if (res->flags & IORESOURCE_IO)
+ pci_unmap_iospace(res);
if (res->parent &&
(res->flags & (IORESOURCE_MEM | IORESOURCE_IO)))
release_resource(res);
--
1.9.1
Rafael J. Wysocki
2016-05-10 18:30:01 UTC
Permalink
Post by Tomasz Nowicki
Platforms that have memory mapped IO port (such as ARM64) need special
handling for PCI I/O resources. For host bridge's resource probing case
these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.
The same I/O resources need to be released after hotplug
removal so that it can be re-added back by the pci_remap_iospace
function during insertion. As a consequence we unmap I/O resources
with pci_unmap_iospace when we release host bridge resources.
---
drivers/acpi/pci_root.c | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index ae3fe4e..cb3071d 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
resource_list_add_tail(entry, resources);
}
}
+static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
+{
+#ifdef PCI_IOBASE
Same comment about the #ifdefs as in the other patch.
Post by Tomasz Nowicki
+ struct resource *res = entry->res;
+ resource_size_t cpu_addr = res->start;
+ resource_size_t pci_addr = cpu_addr - entry->offset;
+ resource_size_t length = resource_size(res);
+ unsigned long port;
+
+ if (pci_register_io_range(cpu_addr, length))
+ goto err;
+
+ port = pci_address_to_pio(cpu_addr);
+ if (port == (unsigned long)-1)
+ goto err;
+
+ res->start = port;
+ res->end = port + length - 1;
+ entry->offset = port - pci_addr;
+
+ if (pci_remap_iospace(res, cpu_addr) < 0)
+ goto err;
An empty line here?
Post by Tomasz Nowicki
+ pr_info("Remapped I/O %pa to %pR\n", &cpu_addr, res);
+ return;
+ res->flags |= IORESOURCE_DISABLED;
+#endif
+}
int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
{
@@ -740,6 +768,9 @@ int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info)
"no IO and memory resources present in _CRS\n");
else {
resource_list_for_each_entry_safe(entry, tmp, list) {
+ if (entry->res->flags & IORESOURCE_IO)
+ acpi_pci_root_remap_iospace(entry);
+
if (entry->res->flags & IORESOURCE_DISABLED)
resource_list_destroy_entry(entry);
else
@@ -811,6 +842,8 @@ static void acpi_pci_root_release_info(struct pci_host_bridge *bridge)
resource_list_for_each_entry(entry, &bridge->windows) {
res = entry->res;
+ if (res->flags & IORESOURCE_IO)
+ pci_unmap_iospace(res);
if (res->parent &&
(res->flags & (IORESOURCE_MEM | IORESOURCE_IO)))
release_resource(res);
--
Tomasz Nowicki
2016-05-11 07:40:02 UTC
Permalink
Post by Rafael J. Wysocki
Post by Tomasz Nowicki
Platforms that have memory mapped IO port (such as ARM64) need special
handling for PCI I/O resources. For host bridge's resource probing case
these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.
The same I/O resources need to be released after hotplug
removal so that it can be re-added back by the pci_remap_iospace
function during insertion. As a consequence we unmap I/O resources
with pci_unmap_iospace when we release host bridge resources.
---
drivers/acpi/pci_root.c | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index ae3fe4e..cb3071d 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
resource_list_add_tail(entry, resources);
}
}
+static void acpi_pci_root_remap_iospace(struct resource_entry *entry)
+{
+#ifdef PCI_IOBASE
Same comment about the #ifdefs as in the other patch.
OK
Post by Rafael J. Wysocki
Post by Tomasz Nowicki
+ struct resource *res = entry->res;
+ resource_size_t cpu_addr = res->start;
+ resource_size_t pci_addr = cpu_addr - entry->offset;
+ resource_size_t length = resource_size(res);
+ unsigned long port;
+
+ if (pci_register_io_range(cpu_addr, length))
+ goto err;
+
+ port = pci_address_to_pio(cpu_addr);
+ if (port == (unsigned long)-1)
+ goto err;
+
+ res->start = port;
+ res->end = port + length - 1;
+ entry->offset = port - pci_addr;
+
+ if (pci_remap_iospace(res, cpu_addr) < 0)
+ goto err;
An empty line here?
yes, empty line would be nice here.

Tomasz
Tomasz Nowicki
2016-05-10 15:30:03 UTC
Permalink
From: Jayachandran C <***@broadcom.com>

Add config option PCI_ECAM and file drivers/pci/ecam.c to provide
generic functions for accessing memory mapped PCI config space.

The API is defined in drivers/pci/ecam.h and is written to replace the
API in drivers/pci/host/pci-host-common.h. The file defines a new
'struct pci_config_window' to hold the information related to a PCI
config area and its mapping. This structure is expected to be used as
sysdata for controllers that have ECAM based mapping.

Helper functions are provided to setup the mapping, free the mapping
and to implement the map_bus method in 'struct pci_ops'

Signed-off-by: Jayachandran C <***@broadcom.com>
---
drivers/pci/Kconfig | 3 +
drivers/pci/Makefile | 2 +
drivers/pci/ecam.c | 161 +++++++++++++++++++++++++++++++++++++++++++++++++++
drivers/pci/ecam.h | 62 ++++++++++++++++++++
4 files changed, 228 insertions(+)
create mode 100644 drivers/pci/ecam.c
create mode 100644 drivers/pci/ecam.h

diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index 209292e..56389be 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -83,6 +83,9 @@ config HT_IRQ
config PCI_ATS
bool

+config PCI_ECAM
+ bool
+
config PCI_IOV
bool "PCI IOV support"
depends on PCI
diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
index 2154092..1fa6925 100644
--- a/drivers/pci/Makefile
+++ b/drivers/pci/Makefile
@@ -55,6 +55,8 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o

obj-$(CONFIG_PCI_STUB) += pci-stub.o

+obj-$(CONFIG_PCI_ECAM) += ecam.o
+
obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o

obj-$(CONFIG_OF) += of.o
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c
new file mode 100644
index 0000000..3d52005
--- /dev/null
+++ b/drivers/pci/ecam.c
@@ -0,0 +1,161 @@
+/*
+ * Copyright 2016 Broadcom
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+
+#include <linux/device.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/slab.h>
+
+#include "ecam.h"
+
+/*
+ * On 64 bit systems, we do a single ioremap for the whole config space
+ * since we have enough virtual address range available. On 32 bit, do an
+ * ioremap per bus.
+ */
+static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);
+
+/*
+ * Create a PCI config space window
+ * - reserve mem region
+ * - alloc struct pci_config_window with space for all mappings
+ * - ioremap the config space
+ */
+struct pci_config_window *pci_ecam_create(struct device *dev,
+ struct resource *cfgres, struct resource *busr,
+ struct pci_ecam_ops *ops)
+{
+ struct pci_config_window *cfg;
+ unsigned int bus_range, bus_range_max, bsz;
+ int i, err;
+
+ if (busr->start > busr->end)
+ return ERR_PTR(-EINVAL);
+
+ cfg = kzalloc(sizeof(*cfg), GFP_KERNEL);
+ if (!cfg)
+ return ERR_PTR(-ENOMEM);
+
+ cfg->ops = ops;
+ cfg->busr.start = busr->start;
+ cfg->busr.end = busr->end;
+ cfg->busr.flags = IORESOURCE_BUS;
+ bus_range = resource_size(&cfg->busr);
+ bus_range_max = resource_size(cfgres) >> ops->bus_shift;
+ if (bus_range > bus_range_max) {
+ dev_warn(dev, "bus max %#x reduced to %#x",
+ bus_range, bus_range_max);
+ bus_range = bus_range_max;
+ cfg->busr.end = busr->start + bus_range - 1;
+ }
+ bsz = 1 << ops->bus_shift;
+
+ cfg->res.start = cfgres->start;
+ cfg->res.end = cfgres->end;
+ cfg->res.flags = IORESOURCE_MEM | IORESOURCE_BUSY;
+ cfg->res.name = "PCI ECAM";
+
+ err = request_resource(&iomem_resource, &cfg->res);
+ if (err) {
+ dev_err(dev, "request ECAM res %pR failed\n", &cfg->res);
+ goto err_exit;
+ }
+
+ if (per_bus_mapping) {
+ cfg->winp = kcalloc(bus_range, sizeof(*cfg->winp), GFP_KERNEL);
+ if (!cfg->winp)
+ goto err_exit_malloc;
+ for (i = 0; i < bus_range; i++) {
+ cfg->winp[i] = ioremap(cfgres->start + i * bsz, bsz);
+ if (!cfg->winp[i])
+ goto err_exit_iomap;
+ }
+ } else {
+ cfg->win = ioremap(cfgres->start, bus_range * bsz);
+ if (!cfg->win)
+ goto err_exit_iomap;
+ }
+
+ if (ops->init) {
+ err = ops->init(dev, cfg);
+ if (err)
+ goto err_exit;
+ }
+ dev_info(dev, "ECAM at %pR for %pR\n", &cfg->res, &cfg->busr);
+ return cfg;
+
+err_exit_iomap:
+ dev_err(dev, "ECAM ioremap failed\n");
+err_exit_malloc:
+ err = -ENOMEM;
+err_exit:
+ pci_ecam_free(cfg);
+ return ERR_PTR(err);
+}
+
+void pci_ecam_free(struct pci_config_window *cfg)
+{
+ int i;
+
+ if (per_bus_mapping) {
+ if (cfg->winp) {
+ for (i = 0; i < resource_size(&cfg->busr); i++)
+ if (cfg->winp[i])
+ iounmap(cfg->winp[i]);
+ kfree(cfg->winp);
+ }
+ } else {
+ if (cfg->win)
+ iounmap(cfg->win);
+ }
+ if (cfg->res.parent)
+ release_resource(&cfg->res);
+ kfree(cfg);
+}
+
+/*
+ * Function to implement the pci_ops ->map_bus method
+ */
+void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
+ int where)
+{
+ struct pci_config_window *cfg = bus->sysdata;
+ unsigned int devfn_shift = cfg->ops->bus_shift - 8;
+ unsigned int busn = bus->number;
+ void __iomem *base;
+
+ if (busn < cfg->busr.start || busn > cfg->busr.end)
+ return NULL;
+
+ busn -= cfg->busr.start;
+ if (per_bus_mapping)
+ base = cfg->winp[busn];
+ else
+ base = cfg->win + (busn << cfg->ops->bus_shift);
+ return base + (devfn << devfn_shift) + where;
+}
+
+/* ECAM ops */
+struct pci_ecam_ops pci_generic_ecam_ops = {
+ .bus_shift = 20,
+ .pci_ops = {
+ .map_bus = pci_ecam_map_bus,
+ .read = pci_generic_config_read,
+ .write = pci_generic_config_write,
+ }
+};
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
new file mode 100644
index 0000000..57de00d
--- /dev/null
+++ b/drivers/pci/ecam.h
@@ -0,0 +1,62 @@
+/*
+ * Copyright 2016 Broadcom
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License, version 2, as
+ * published by the Free Software Foundation (the "GPL").
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License version 2 (GPLv2) for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * version 2 (GPLv2) along with this source code.
+ */
+#ifndef DRIVERS_PCI_ECAM_H
+#define DRIVERS_PCI_ECAM_H
+
+#include <linux/kernel.h>
+#include <linux/platform_device.h>
+
+/*
+ * struct to hold pci ops and bus shift of the config window
+ * for a PCI controller.
+ */
+struct pci_config_window;
+struct pci_ecam_ops {
+ unsigned int bus_shift;
+ struct pci_ops pci_ops;
+ int (*init)(struct device *,
+ struct pci_config_window *);
+};
+
+/*
+ * struct to hold the mappings of a config space window. This
+ * is expected to be used as sysdata for PCI controlllers which
+ * use ECAM.
+ */
+struct pci_config_window {
+ struct resource res;
+ struct resource busr;
+ void *priv;
+ struct pci_ecam_ops *ops;
+ union {
+ void __iomem *win; /* 64-bit single mapping */
+ void __iomem **winp; /* 32-bit per bus mapping */
+ };
+};
+
+/* create and free for pci_config_window */
+struct pci_config_window *pci_ecam_create(struct device *dev,
+ struct resource *cfgres, struct resource *busr,
+ struct pci_ecam_ops *ops);
+void pci_ecam_free(struct pci_config_window *cfg);
+
+/* map_bus when ->sysdata is an instance of pci_config_window */
+void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
+ int where);
+/* default ECAM ops, bus shift 20, generic read and write */
+extern struct pci_ecam_ops pci_generic_ecam_ops;
+
+#endif
--
1.9.1
Tomasz Nowicki
2016-05-10 15:30:03 UTC
Permalink
No functional changes in this patch.

PCI I/O space mapping code does not depend on OF, therefore it can be
moved to PCI core code. This way we will be able to use it
e.g. in ACPI PCI code.

Suggested-by: Lorenzo Pieralisi <***@arm.com>
Signed-off-by: Tomasz Nowicki <***@semihalf.com>
CC: Arnd Bergmann <***@arndb.de>
CC: Liviu Dudau <***@arm.com>
---
drivers/of/address.c | 116 +--------------------------------------------
drivers/pci/pci.c | 115 ++++++++++++++++++++++++++++++++++++++++++++
include/linux/of_address.h | 9 ----
include/linux/pci.h | 5 ++
4 files changed, 121 insertions(+), 124 deletions(-)

diff --git a/drivers/of/address.c b/drivers/of/address.c
index 91a469d..0a553c0 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -4,6 +4,7 @@
#include <linux/ioport.h>
#include <linux/module.h>
#include <linux/of_address.h>
+#include <linux/pci.h>
#include <linux/pci_regs.h>
#include <linux/sizes.h>
#include <linux/slab.h>
@@ -673,121 +674,6 @@ const __be32 *of_get_address(struct device_node *dev, int index, u64 *size,
}
EXPORT_SYMBOL(of_get_address);

-#ifdef PCI_IOBASE
-struct io_range {
- struct list_head list;
- phys_addr_t start;
- resource_size_t size;
-};
-
-static LIST_HEAD(io_range_list);
-static DEFINE_SPINLOCK(io_range_lock);
-#endif
-
-/*
- * Record the PCI IO range (expressed as CPU physical address + size).
- * Return a negative value if an error has occured, zero otherwise
- */
-int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size)
-{
- int err = 0;
-
-#ifdef PCI_IOBASE
- struct io_range *range;
- resource_size_t allocated_size = 0;
-
- /* check if the range hasn't been previously recorded */
- spin_lock(&io_range_lock);
- list_for_each_entry(range, &io_range_list, list) {
- if (addr >= range->start && addr + size <= range->start + size) {
- /* range already registered, bail out */
- goto end_register;
- }
- allocated_size += range->size;
- }
-
- /* range not registed yet, check for available space */
- if (allocated_size + size - 1 > IO_SPACE_LIMIT) {
- /* if it's too big check if 64K space can be reserved */
- if (allocated_size + SZ_64K - 1 > IO_SPACE_LIMIT) {
- err = -E2BIG;
- goto end_register;
- }
-
- size = SZ_64K;
- pr_warn("Requested IO range too big, new size set to 64K\n");
- }
-
- /* add the range to the list */
- range = kzalloc(sizeof(*range), GFP_ATOMIC);
- if (!range) {
- err = -ENOMEM;
- goto end_register;
- }
-
- range->start = addr;
- range->size = size;
-
- list_add_tail(&range->list, &io_range_list);
-
-end_register:
- spin_unlock(&io_range_lock);
-#endif
-
- return err;
-}
-
-phys_addr_t pci_pio_to_address(unsigned long pio)
-{
- phys_addr_t address = (phys_addr_t)OF_BAD_ADDR;
-
-#ifdef PCI_IOBASE
- struct io_range *range;
- resource_size_t allocated_size = 0;
-
- if (pio > IO_SPACE_LIMIT)
- return address;
-
- spin_lock(&io_range_lock);
- list_for_each_entry(range, &io_range_list, list) {
- if (pio >= allocated_size && pio < allocated_size + range->size) {
- address = range->start + pio - allocated_size;
- break;
- }
- allocated_size += range->size;
- }
- spin_unlock(&io_range_lock);
-#endif
-
- return address;
-}
-
-unsigned long __weak pci_address_to_pio(phys_addr_t address)
-{
-#ifdef PCI_IOBASE
- struct io_range *res;
- resource_size_t offset = 0;
- unsigned long addr = -1;
-
- spin_lock(&io_range_lock);
- list_for_each_entry(res, &io_range_list, list) {
- if (address >= res->start && address < res->start + res->size) {
- addr = address - res->start + offset;
- break;
- }
- offset += res->size;
- }
- spin_unlock(&io_range_lock);
-
- return addr;
-#else
- if (address > IO_SPACE_LIMIT)
- return (unsigned long)-1;
-
- return (unsigned long) address;
-#endif
-}
-
static int __of_address_to_resource(struct device_node *dev,
const __be32 *addrp, u64 size, unsigned int flags,
const char *name, struct resource *r)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 25e0327..bc0c914 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3021,6 +3021,121 @@ int pci_request_regions_exclusive(struct pci_dev *pdev, const char *res_name)
}
EXPORT_SYMBOL(pci_request_regions_exclusive);

+#ifdef PCI_IOBASE
+struct io_range {
+ struct list_head list;
+ phys_addr_t start;
+ resource_size_t size;
+};
+
+static LIST_HEAD(io_range_list);
+static DEFINE_SPINLOCK(io_range_lock);
+#endif
+
+/*
+ * Record the PCI IO range (expressed as CPU physical address + size).
+ * Return a negative value if an error has occured, zero otherwise
+ */
+int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size)
+{
+ int err = 0;
+
+#ifdef PCI_IOBASE
+ struct io_range *range;
+ resource_size_t allocated_size = 0;
+
+ /* check if the range hasn't been previously recorded */
+ spin_lock(&io_range_lock);
+ list_for_each_entry(range, &io_range_list, list) {
+ if (addr >= range->start && addr + size <= range->start + size) {
+ /* range already registered, bail out */
+ goto end_register;
+ }
+ allocated_size += range->size;
+ }
+
+ /* range not registed yet, check for available space */
+ if (allocated_size + size - 1 > IO_SPACE_LIMIT) {
+ /* if it's too big check if 64K space can be reserved */
+ if (allocated_size + SZ_64K - 1 > IO_SPACE_LIMIT) {
+ err = -E2BIG;
+ goto end_register;
+ }
+
+ size = SZ_64K;
+ pr_warn("Requested IO range too big, new size set to 64K\n");
+ }
+
+ /* add the range to the list */
+ range = kzalloc(sizeof(*range), GFP_ATOMIC);
+ if (!range) {
+ err = -ENOMEM;
+ goto end_register;
+ }
+
+ range->start = addr;
+ range->size = size;
+
+ list_add_tail(&range->list, &io_range_list);
+
+end_register:
+ spin_unlock(&io_range_lock);
+#endif
+
+ return err;
+}
+
+phys_addr_t pci_pio_to_address(unsigned long pio)
+{
+ phys_addr_t address = (phys_addr_t)OF_BAD_ADDR;
+
+#ifdef PCI_IOBASE
+ struct io_range *range;
+ resource_size_t allocated_size = 0;
+
+ if (pio > IO_SPACE_LIMIT)
+ return address;
+
+ spin_lock(&io_range_lock);
+ list_for_each_entry(range, &io_range_list, list) {
+ if (pio >= allocated_size && pio < allocated_size + range->size) {
+ address = range->start + pio - allocated_size;
+ break;
+ }
+ allocated_size += range->size;
+ }
+ spin_unlock(&io_range_lock);
+#endif
+
+ return address;
+}
+
+unsigned long __weak pci_address_to_pio(phys_addr_t address)
+{
+#ifdef PCI_IOBASE
+ struct io_range *res;
+ resource_size_t offset = 0;
+ unsigned long addr = -1;
+
+ spin_lock(&io_range_lock);
+ list_for_each_entry(res, &io_range_list, list) {
+ if (address >= res->start && address < res->start + res->size) {
+ addr = address - res->start + offset;
+ break;
+ }
+ offset += res->size;
+ }
+ spin_unlock(&io_range_lock);
+
+ return addr;
+#else
+ if (address > IO_SPACE_LIMIT)
+ return (unsigned long)-1;
+
+ return (unsigned long) address;
+#endif
+}
+
/**
* pci_remap_iospace - Remap the memory mapped I/O space
* @res: Resource describing the I/O space
diff --git a/include/linux/of_address.h b/include/linux/of_address.h
index 01c0a55..3786473 100644
--- a/include/linux/of_address.h
+++ b/include/linux/of_address.h
@@ -47,10 +47,6 @@ void __iomem *of_io_request_and_map(struct device_node *device,
extern const __be32 *of_get_address(struct device_node *dev, int index,
u64 *size, unsigned int *flags);

-extern int pci_register_io_range(phys_addr_t addr, resource_size_t size);
-extern unsigned long pci_address_to_pio(phys_addr_t addr);
-extern phys_addr_t pci_pio_to_address(unsigned long pio);
-
extern int of_pci_range_parser_init(struct of_pci_range_parser *parser,
struct device_node *node);
extern struct of_pci_range *of_pci_range_parser_one(
@@ -86,11 +82,6 @@ static inline const __be32 *of_get_address(struct device_node *dev, int index,
return NULL;
}

-static inline phys_addr_t pci_pio_to_address(unsigned long pio)
-{
- return 0;
-}
-
static inline int of_pci_range_parser_init(struct of_pci_range_parser *parser,
struct device_node *node)
{
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 932ec74..d2a57f3 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1165,6 +1165,9 @@ int __must_check pci_bus_alloc_resource(struct pci_bus *bus,
void *alignf_data);


+int pci_register_io_range(phys_addr_t addr, resource_size_t size);
+unsigned long pci_address_to_pio(phys_addr_t addr);
+phys_addr_t pci_pio_to_address(unsigned long pio);
int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr);

static inline pci_bus_addr_t pci_bus_address(struct pci_dev *pdev, int bar)
@@ -1481,6 +1484,8 @@ static inline int pci_request_regions(struct pci_dev *dev, const char *res_name)
{ return -EIO; }
static inline void pci_release_regions(struct pci_dev *dev) { }

+static inline unsigned long pci_address_to_pio(phys_addr_t addr) { return -1; }
+
static inline void pci_block_cfg_access(struct pci_dev *dev) { }
static inline int pci_block_cfg_access_in_atomic(struct pci_dev *dev)
{ return 0; }
--
1.9.1
Rafael J. Wysocki
2016-05-10 18:00:02 UTC
Permalink
Post by Tomasz Nowicki
No functional changes in this patch.
PCI I/O space mapping code does not depend on OF, therefore it can be
moved to PCI core code. This way we will be able to use it
e.g. in ACPI PCI code.
---
drivers/of/address.c | 116 +--------------------------------------------
drivers/pci/pci.c | 115 ++++++++++++++++++++++++++++++++++++++++++++
include/linux/of_address.h | 9 ----
include/linux/pci.h | 5 ++
4 files changed, 121 insertions(+), 124 deletions(-)
diff --git a/drivers/of/address.c b/drivers/of/address.c
index 91a469d..0a553c0 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -4,6 +4,7 @@
#include <linux/ioport.h>
#include <linux/module.h>
#include <linux/of_address.h>
+#include <linux/pci.h>
#include <linux/pci_regs.h>
#include <linux/sizes.h>
#include <linux/slab.h>
@@ -673,121 +674,6 @@ const __be32 *of_get_address(struct device_node *dev, int index, u64 *size,
}
EXPORT_SYMBOL(of_get_address);
-#ifdef PCI_IOBASE
-struct io_range {
- struct list_head list;
- phys_addr_t start;
- resource_size_t size;
-};
-
-static LIST_HEAD(io_range_list);
-static DEFINE_SPINLOCK(io_range_lock);
-#endif
-
-/*
- * Record the PCI IO range (expressed as CPU physical address + size).
- * Return a negative value if an error has occured, zero otherwise
- */
-int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size)
-{
- int err = 0;
-
-#ifdef PCI_IOBASE
- struct io_range *range;
- resource_size_t allocated_size = 0;
-
- /* check if the range hasn't been previously recorded */
- spin_lock(&io_range_lock);
- list_for_each_entry(range, &io_range_list, list) {
- if (addr >= range->start && addr + size <= range->start + size) {
- /* range already registered, bail out */
- goto end_register;
- }
- allocated_size += range->size;
- }
-
- /* range not registed yet, check for available space */
- if (allocated_size + size - 1 > IO_SPACE_LIMIT) {
- /* if it's too big check if 64K space can be reserved */
- if (allocated_size + SZ_64K - 1 > IO_SPACE_LIMIT) {
- err = -E2BIG;
- goto end_register;
- }
-
- size = SZ_64K;
- pr_warn("Requested IO range too big, new size set to 64K\n");
- }
-
- /* add the range to the list */
- range = kzalloc(sizeof(*range), GFP_ATOMIC);
- if (!range) {
- err = -ENOMEM;
- goto end_register;
- }
-
- range->start = addr;
- range->size = size;
-
- list_add_tail(&range->list, &io_range_list);
-
- spin_unlock(&io_range_lock);
-#endif
-
- return err;
-}
-
-phys_addr_t pci_pio_to_address(unsigned long pio)
-{
- phys_addr_t address = (phys_addr_t)OF_BAD_ADDR;
-
-#ifdef PCI_IOBASE
- struct io_range *range;
- resource_size_t allocated_size = 0;
-
- if (pio > IO_SPACE_LIMIT)
- return address;
-
- spin_lock(&io_range_lock);
- list_for_each_entry(range, &io_range_list, list) {
- if (pio >= allocated_size && pio < allocated_size + range->size) {
- address = range->start + pio - allocated_size;
- break;
- }
- allocated_size += range->size;
- }
- spin_unlock(&io_range_lock);
-#endif
-
- return address;
-}
-
-unsigned long __weak pci_address_to_pio(phys_addr_t address)
-{
-#ifdef PCI_IOBASE
- struct io_range *res;
- resource_size_t offset = 0;
- unsigned long addr = -1;
-
- spin_lock(&io_range_lock);
- list_for_each_entry(res, &io_range_list, list) {
- if (address >= res->start && address < res->start + res->size) {
- addr = address - res->start + offset;
- break;
- }
- offset += res->size;
- }
- spin_unlock(&io_range_lock);
-
- return addr;
-#else
- if (address > IO_SPACE_LIMIT)
- return (unsigned long)-1;
-
- return (unsigned long) address;
-#endif
-}
-
static int __of_address_to_resource(struct device_node *dev,
const __be32 *addrp, u64 size, unsigned int flags,
const char *name, struct resource *r)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 25e0327..bc0c914 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3021,6 +3021,121 @@ int pci_request_regions_exclusive(struct pci_dev *pdev, const char *res_name)
}
EXPORT_SYMBOL(pci_request_regions_exclusive);
+#ifdef PCI_IOBASE
+struct io_range {
+ struct list_head list;
+ phys_addr_t start;
+ resource_size_t size;
+};
+
+static LIST_HEAD(io_range_list);
+static DEFINE_SPINLOCK(io_range_lock);
+#endif
+
+/*
+ * Record the PCI IO range (expressed as CPU physical address + size).
+ * Return a negative value if an error has occured, zero otherwise
+ */
+int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size)
+{
+ int err = 0;
+
+#ifdef PCI_IOBASE
I understand that this moves code around, but those in-function
#ifdefs aren't nice. Any chance to get rid of them but putting whole
functions under the #ifdef?
Tomasz Nowicki
2016-05-11 07:40:02 UTC
Permalink
Post by Rafael J. Wysocki
Post by Tomasz Nowicki
No functional changes in this patch.
PCI I/O space mapping code does not depend on OF, therefore it can be
moved to PCI core code. This way we will be able to use it
e.g. in ACPI PCI code.
---
drivers/of/address.c | 116 +--------------------------------------------
drivers/pci/pci.c | 115 ++++++++++++++++++++++++++++++++++++++++++++
include/linux/of_address.h | 9 ----
include/linux/pci.h | 5 ++
4 files changed, 121 insertions(+), 124 deletions(-)
diff --git a/drivers/of/address.c b/drivers/of/address.c
index 91a469d..0a553c0 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -4,6 +4,7 @@
#include <linux/ioport.h>
#include <linux/module.h>
#include <linux/of_address.h>
+#include <linux/pci.h>
#include <linux/pci_regs.h>
#include <linux/sizes.h>
#include <linux/slab.h>
@@ -673,121 +674,6 @@ const __be32 *of_get_address(struct device_node *dev, int index, u64 *size,
}
EXPORT_SYMBOL(of_get_address);
-#ifdef PCI_IOBASE
-struct io_range {
- struct list_head list;
- phys_addr_t start;
- resource_size_t size;
-};
-
-static LIST_HEAD(io_range_list);
-static DEFINE_SPINLOCK(io_range_lock);
-#endif
-
-/*
- * Record the PCI IO range (expressed as CPU physical address + size).
- * Return a negative value if an error has occured, zero otherwise
- */
-int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size)
-{
- int err = 0;
-
-#ifdef PCI_IOBASE
- struct io_range *range;
- resource_size_t allocated_size = 0;
-
- /* check if the range hasn't been previously recorded */
- spin_lock(&io_range_lock);
- list_for_each_entry(range, &io_range_list, list) {
- if (addr >= range->start && addr + size <= range->start + size) {
- /* range already registered, bail out */
- goto end_register;
- }
- allocated_size += range->size;
- }
-
- /* range not registed yet, check for available space */
- if (allocated_size + size - 1 > IO_SPACE_LIMIT) {
- /* if it's too big check if 64K space can be reserved */
- if (allocated_size + SZ_64K - 1 > IO_SPACE_LIMIT) {
- err = -E2BIG;
- goto end_register;
- }
-
- size = SZ_64K;
- pr_warn("Requested IO range too big, new size set to 64K\n");
- }
-
- /* add the range to the list */
- range = kzalloc(sizeof(*range), GFP_ATOMIC);
- if (!range) {
- err = -ENOMEM;
- goto end_register;
- }
-
- range->start = addr;
- range->size = size;
-
- list_add_tail(&range->list, &io_range_list);
-
- spin_unlock(&io_range_lock);
-#endif
-
- return err;
-}
-
-phys_addr_t pci_pio_to_address(unsigned long pio)
-{
- phys_addr_t address = (phys_addr_t)OF_BAD_ADDR;
-
-#ifdef PCI_IOBASE
- struct io_range *range;
- resource_size_t allocated_size = 0;
-
- if (pio > IO_SPACE_LIMIT)
- return address;
-
- spin_lock(&io_range_lock);
- list_for_each_entry(range, &io_range_list, list) {
- if (pio >= allocated_size && pio < allocated_size + range->size) {
- address = range->start + pio - allocated_size;
- break;
- }
- allocated_size += range->size;
- }
- spin_unlock(&io_range_lock);
-#endif
-
- return address;
-}
-
-unsigned long __weak pci_address_to_pio(phys_addr_t address)
-{
-#ifdef PCI_IOBASE
- struct io_range *res;
- resource_size_t offset = 0;
- unsigned long addr = -1;
-
- spin_lock(&io_range_lock);
- list_for_each_entry(res, &io_range_list, list) {
- if (address >= res->start && address < res->start + res->size) {
- addr = address - res->start + offset;
- break;
- }
- offset += res->size;
- }
- spin_unlock(&io_range_lock);
-
- return addr;
-#else
- if (address > IO_SPACE_LIMIT)
- return (unsigned long)-1;
-
- return (unsigned long) address;
-#endif
-}
-
static int __of_address_to_resource(struct device_node *dev,
const __be32 *addrp, u64 size, unsigned int flags,
const char *name, struct resource *r)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 25e0327..bc0c914 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -3021,6 +3021,121 @@ int pci_request_regions_exclusive(struct pci_dev *pdev, const char *res_name)
}
EXPORT_SYMBOL(pci_request_regions_exclusive);
+#ifdef PCI_IOBASE
+struct io_range {
+ struct list_head list;
+ phys_addr_t start;
+ resource_size_t size;
+};
+
+static LIST_HEAD(io_range_list);
+static DEFINE_SPINLOCK(io_range_lock);
+#endif
+
+/*
+ * Record the PCI IO range (expressed as CPU physical address + size).
+ * Return a negative value if an error has occured, zero otherwise
+ */
+int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size)
+{
+ int err = 0;
+
+#ifdef PCI_IOBASE
I understand that this moves code around, but those in-function
#ifdefs aren't nice. Any chance to get rid of them but putting whole
functions under the #ifdef?
This is a __weak implementation, so assuming I would move #ifdef out of
function I need to provide another empty __weak stub. I do not know
which solution is more ugly. In any case we can do that cleanup separately.

Tomasz
Arnd Bergmann
2016-05-11 11:10:03 UTC
Permalink
Post by Tomasz Nowicki
Post by Rafael J. Wysocki
I understand that this moves code around, but those in-function
#ifdefs aren't nice. Any chance to get rid of them but putting whole
functions under the #ifdef?
This is a __weak implementation, so assuming I would move #ifdef out of
function I need to provide another empty __weak stub. I do not know
which solution is more ugly. In any case we can do that cleanup separately.
I'd vote for just dropping the __weak here, given that there is no
non-weak implementation. If we end up needing to override this for
some architecture or host bridge in the future, we can think about how
to best do that then.

I agree that should be a separate patch, this one should only move
code from one file to another.

Arnd
Tomasz Nowicki
2016-05-10 15:30:03 UTC
Permalink
It is very useful to release I/O resources so that the same I/O resources
can be allocated again (pci_remap_iospace), like in PCI hotplug removal
scenario. Therefore this patch implements new pci_unmap_iospace call which
unmaps I/O space as the symmetry to pci_remap_iospace.

Signed-off-by: Sinan Kaya <***@codeaurora.org>
Signed-off-by: Tomasz Nowicki <***@semihalf.com>
---
drivers/pci/pci.c | 24 ++++++++++++++++++++++++
include/linux/pci.h | 1 +
2 files changed, 25 insertions(+)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index bc0c914..ff97a0b 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -25,6 +25,7 @@
#include <linux/device.h>
#include <linux/pm_runtime.h>
#include <linux/pci_hotplug.h>
+#include <linux/vmalloc.h>
#include <asm/setup.h>
#include <linux/aer.h>
#include "pci.h"
@@ -3167,6 +3168,29 @@ int __weak pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr)
#endif
}

+/**
+ * pci_unmap_iospace - Unmap the memory mapped I/O space
+ * @res: resource to be unmapped
+ *
+ * Unmap the CPU virtual address @res from virtual address space.
+ * Only architectures that have memory mapped IO functions defined
+ * (and the PCI_IOBASE value defined) should call this function.
+ */
+void pci_unmap_iospace(struct resource *res)
+{
+#if defined(PCI_IOBASE) && defined(CONFIG_MMU)
+ unsigned long vaddr = (unsigned long)PCI_IOBASE + res->start;
+
+ unmap_kernel_range(vaddr, resource_size(res));
+#else
+ /*
+ * This architecture does not have memory mapped I/O space,
+ * so this function should never be called.
+ */
+ WARN_ONCE(1, "This architecture does not support memory mapped I/O\n");
+#endif
+}
+
static void __pci_set_master(struct pci_dev *dev, bool enable)
{
u16 old_cmd, cmd;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index d2a57f3..d6ea6ce 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1169,6 +1169,7 @@ int pci_register_io_range(phys_addr_t addr, resource_size_t size);
unsigned long pci_address_to_pio(phys_addr_t addr);
phys_addr_t pci_pio_to_address(unsigned long pio);
int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr);
+void pci_unmap_iospace(struct resource *res);

static inline pci_bus_addr_t pci_bus_address(struct pci_dev *pdev, int bar)
{
--
1.9.1
Jayachandran C
2016-05-23 08:30:01 UTC
Permalink
Post by Tomasz Nowicki
It is very useful to release I/O resources so that the same I/O resources
can be allocated again (pci_remap_iospace), like in PCI hotplug removal
scenario. Therefore this patch implements new pci_unmap_iospace call which
unmaps I/O space as the symmetry to pci_remap_iospace.
---
drivers/pci/pci.c | 24 ++++++++++++++++++++++++
include/linux/pci.h | 1 +
2 files changed, 25 insertions(+)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index bc0c914..ff97a0b 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -25,6 +25,7 @@
#include <linux/device.h>
#include <linux/pm_runtime.h>
#include <linux/pci_hotplug.h>
+#include <linux/vmalloc.h>
#include <asm/setup.h>
#include <linux/aer.h>
#include "pci.h"
@@ -3167,6 +3168,29 @@ int __weak pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr)
#endif
}
+/**
+ * pci_unmap_iospace - Unmap the memory mapped I/O space
+ *
+ * Only architectures that have memory mapped IO functions defined
+ * (and the PCI_IOBASE value defined) should call this function.
+ */
+void pci_unmap_iospace(struct resource *res)
+{
+#if defined(PCI_IOBASE) && defined(CONFIG_MMU)
+ unsigned long vaddr = (unsigned long)PCI_IOBASE + res->start;
+
+ unmap_kernel_range(vaddr, resource_size(res));
+#else
+ /*
+ * This architecture does not have memory mapped I/O space,
+ * so this function should never be called.
+ */
+ WARN_ONCE(1, "This architecture does not support memory mapped I/O\n");
+#endif
+}
WARN is not needed here, since we would have already done it in
pci_remap_iospace.

Ideally, we should undo the pci_register_io_range as well, but
re-registering the same range seems to be fine.

JC.
Tomasz Nowicki
2016-05-10 15:30:03 UTC
Permalink
It is perfectly fine to use ACPI_PCI_HOST_GENERIC for ARM64,
so lets get rid of PCI init and RAW ACPI accessor empty stubs
and go with full-blown PCI host controller driver.

Signed-off-by: Tomasz Nowicki <***@semihalf.com>
To: Catalin Marinas <***@arm.com>
To: Lorenzo Pieralisi <***@arm.com>
To: Will Deacon <***@arm.com>
To: Arnd Bergmann <***@arndb.de>
---
arch/arm64/Kconfig | 1 +
arch/arm64/kernel/pci.c | 24 ------------------------
2 files changed, 1 insertion(+), 24 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 4f43622..1bded87 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -2,6 +2,7 @@ config ARM64
def_bool y
select ACPI_CCA_REQUIRED if ACPI
select ACPI_GENERIC_GSI if ACPI
+ select ACPI_PCI_HOST_GENERIC if ACPI
select ACPI_REDUCED_HARDWARE_ONLY if ACPI
select ARCH_HAS_DEVMEM_IS_ALLOWED
select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index eeec5f6..e484c91 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -74,27 +74,3 @@ void pcibios_remove_bus(struct pci_bus *bus)
{
acpi_pci_remove_bus(bus);
}
-
-/*
- * raw_pci_read/write - Platform-specific PCI config space access.
- */
-int raw_pci_read(unsigned int domain, unsigned int bus,
- unsigned int devfn, int reg, int len, u32 *val)
-{
- return -ENXIO;
-}
-
-int raw_pci_write(unsigned int domain, unsigned int bus,
- unsigned int devfn, int reg, int len, u32 val)
-{
- return -ENXIO;
-}
-
-#ifdef CONFIG_ACPI
-/* Root bridge scanning */
-struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
-{
- /* TODO: Should be revisited when implementing PCI on ACPI */
- return NULL;
-}
-#endif
--
1.9.1
Tomasz Nowicki
2016-05-10 15:30:03 UTC
Permalink
From: Jayachandran C <***@broadcom.com>

Use functions provided by drivers/pci/ecam.h for mapping the config
space in drivers/pci/host/pci-host-common.c, and update its users to
use 'struct pci_config_window' and 'struct pci_ecam_ops'

The changes are mostly to use 'struct pci_config_window' in place of
'struct gen_pci'. Some of the fields of gen_pci were only used
temporarily and can be eliminated by using local variables or function
arguments, these are not carried over to struct pci_config_window.

pci-thunder-ecam.c and pci-thunder-pem.c are the only users of the
pci_host_common_probe function and the gen_pci structure, these have
been updated to use the new API as well.

The patch does not introduce any functional changes other than a very
minor one: with the new code, on 64-bit platforms, we do just a single
ioremap for the whole config space.

Signed-off-by: Jayachandran C <***@broadcom.com>
---
drivers/pci/ecam.h | 5 ++
drivers/pci/host/Kconfig | 1 +
drivers/pci/host/pci-host-common.c | 114 +++++++++++++++---------------------
drivers/pci/host/pci-host-common.h | 47 ---------------
drivers/pci/host/pci-host-generic.c | 52 ++++------------
drivers/pci/host/pci-thunder-ecam.c | 39 +++---------
drivers/pci/host/pci-thunder-pem.c | 92 ++++++++++++++---------------
7 files changed, 113 insertions(+), 237 deletions(-)
delete mode 100644 drivers/pci/host/pci-host-common.h

diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h
index 57de00d..1ad2176 100644
--- a/drivers/pci/ecam.h
+++ b/drivers/pci/ecam.h
@@ -59,4 +59,9 @@ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn,
/* default ECAM ops, bus shift 20, generic read and write */
extern struct pci_ecam_ops pci_generic_ecam_ops;

+#ifdef CONFIG_PCI_HOST_GENERIC
+/* for DT based pci controllers that support ECAM */
+int pci_host_common_probe(struct platform_device *pdev,
+ struct pci_ecam_ops *ops);
+#endif
#endif
diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig
index 7a0780d..963300d 100644
--- a/drivers/pci/host/Kconfig
+++ b/drivers/pci/host/Kconfig
@@ -77,6 +77,7 @@ config PCI_RCAR_GEN2_PCIE

config PCI_HOST_COMMON
bool
+ select PCI_ECAM

config PCI_HOST_GENERIC
bool "Generic PCI host controller"
diff --git a/drivers/pci/host/pci-host-common.c b/drivers/pci/host/pci-host-common.c
index e9f850f..8cba7ab 100644
--- a/drivers/pci/host/pci-host-common.c
+++ b/drivers/pci/host/pci-host-common.c
@@ -22,27 +22,21 @@
#include <linux/of_pci.h>
#include <linux/platform_device.h>

-#include "pci-host-common.h"
+#include "../ecam.h"

-static void gen_pci_release_of_pci_ranges(struct gen_pci *pci)
-{
- pci_free_resource_list(&pci->resources);
-}
-
-static int gen_pci_parse_request_of_pci_ranges(struct gen_pci *pci)
+static int gen_pci_parse_request_of_pci_ranges(struct device *dev,
+ struct list_head *resources, struct resource **bus_range)
{
int err, res_valid = 0;
- struct device *dev = pci->host.dev.parent;
struct device_node *np = dev->of_node;
resource_size_t iobase;
struct resource_entry *win;

- err = of_pci_get_host_bridge_resources(np, 0, 0xff, &pci->resources,
- &iobase);
+ err = of_pci_get_host_bridge_resources(np, 0, 0xff, resources, &iobase);
if (err)
return err;

- resource_list_for_each_entry(win, &pci->resources) {
+ resource_list_for_each_entry(win, resources) {
struct resource *parent, *res = win->res;

switch (resource_type(res)) {
@@ -60,7 +54,7 @@ static int gen_pci_parse_request_of_pci_ranges(struct gen_pci *pci)
res_valid |= !(res->flags & IORESOURCE_PREFETCH);
break;
case IORESOURCE_BUS:
- pci->cfg.bus_range = res;
+ *bus_range = res;
default:
continue;
}
@@ -79,65 +73,60 @@ static int gen_pci_parse_request_of_pci_ranges(struct gen_pci *pci)
return 0;

out_release_res:
- gen_pci_release_of_pci_ranges(pci);
return err;
}

-static int gen_pci_parse_map_cfg_windows(struct gen_pci *pci)
+static void gen_pci_unmap_cfg(void *ptr)
+{
+ pci_ecam_free((struct pci_config_window *)ptr);
+}
+
+static struct pci_config_window *gen_pci_init(struct device *dev,
+ struct list_head *resources, struct pci_ecam_ops *ops)
{
int err;
- u8 bus_max;
- resource_size_t busn;
- struct resource *bus_range;
- struct device *dev = pci->host.dev.parent;
- struct device_node *np = dev->of_node;
- u32 sz = 1 << pci->cfg.ops->bus_shift;
+ struct resource cfgres;
+ struct resource *bus_range = NULL;
+ struct pci_config_window *cfg;

- err = of_address_to_resource(np, 0, &pci->cfg.res);
+ /* Parse our PCI ranges and request their resources */
+ err = gen_pci_parse_request_of_pci_ranges(dev, resources, &bus_range);
+ if (err)
+ goto err_out;
+
+ err = of_address_to_resource(dev->of_node, 0, &cfgres);
if (err) {
dev_err(dev, "missing \"reg\" property\n");
- return err;
+ goto err_out;
}

- /* Limit the bus-range to fit within reg */
- bus_max = pci->cfg.bus_range->start +
- (resource_size(&pci->cfg.res) >> pci->cfg.ops->bus_shift) - 1;
- pci->cfg.bus_range->end = min_t(resource_size_t,
- pci->cfg.bus_range->end, bus_max);
-
- pci->cfg.win = devm_kcalloc(dev, resource_size(pci->cfg.bus_range),
- sizeof(*pci->cfg.win), GFP_KERNEL);
- if (!pci->cfg.win)
- return -ENOMEM;
-
- /* Map our Configuration Space windows */
- if (!devm_request_mem_region(dev, pci->cfg.res.start,
- resource_size(&pci->cfg.res),
- "Configuration Space"))
- return -ENOMEM;
-
- bus_range = pci->cfg.bus_range;
- for (busn = bus_range->start; busn <= bus_range->end; ++busn) {
- u32 idx = busn - bus_range->start;
-
- pci->cfg.win[idx] = devm_ioremap(dev,
- pci->cfg.res.start + idx * sz,
- sz);
- if (!pci->cfg.win[idx])
- return -ENOMEM;
+ cfg = pci_ecam_create(dev, &cfgres, bus_range, ops);
+ if (IS_ERR(cfg)) {
+ err = PTR_ERR(cfg);
+ goto err_out;
}

- return 0;
+ err = devm_add_action(dev, gen_pci_unmap_cfg, cfg);
+ if (err) {
+ gen_pci_unmap_cfg(cfg);
+ goto err_out;
+ }
+ return cfg;
+
+err_out:
+ pci_free_resource_list(resources);
+ return ERR_PTR(err);
}

int pci_host_common_probe(struct platform_device *pdev,
- struct gen_pci *pci)
+ struct pci_ecam_ops *ops)
{
- int err;
const char *type;
struct device *dev = &pdev->dev;
struct device_node *np = dev->of_node;
struct pci_bus *bus, *child;
+ struct pci_config_window *cfg;
+ struct list_head resources;

type = of_get_property(np, "device_type", NULL);
if (!type || strcmp(type, "pci")) {
@@ -147,29 +136,18 @@ int pci_host_common_probe(struct platform_device *pdev,

of_pci_check_probe_only();

- pci->host.dev.parent = dev;
- INIT_LIST_HEAD(&pci->host.windows);
- INIT_LIST_HEAD(&pci->resources);
-
- /* Parse our PCI ranges and request their resources */
- err = gen_pci_parse_request_of_pci_ranges(pci);
- if (err)
- return err;
-
/* Parse and map our Configuration Space windows */
- err = gen_pci_parse_map_cfg_windows(pci);
- if (err) {
- gen_pci_release_of_pci_ranges(pci);
- return err;
- }
+ INIT_LIST_HEAD(&resources);
+ cfg = gen_pci_init(dev, &resources, ops);
+ if (IS_ERR(cfg))
+ return PTR_ERR(cfg);

/* Do not reassign resources if probe only */
if (!pci_has_flag(PCI_PROBE_ONLY))
pci_add_flags(PCI_REASSIGN_ALL_RSRC | PCI_REASSIGN_ALL_BUS);

-
- bus = pci_scan_root_bus(dev, pci->cfg.bus_range->start,
- &pci->cfg.ops->ops, pci, &pci->resources);
+ bus = pci_scan_root_bus(dev, cfg->busr.start, &ops->pci_ops, cfg,
+ &resources);
if (!bus) {
dev_err(dev, "Scanning rootbus failed");
return -ENODEV;
diff --git a/drivers/pci/host/pci-host-common.h b/drivers/pci/host/pci-host-common.h
deleted file mode 100644
index 09f3fa0..0000000
--- a/drivers/pci/host/pci-host-common.h
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- * Copyright (C) 2014 ARM Limited
- *
- * Author: Will Deacon <***@arm.com>
- */
-
-#ifndef _PCI_HOST_COMMON_H
-#define _PCI_HOST_COMMON_H
-
-#include <linux/kernel.h>
-#include <linux/platform_device.h>
-
-struct gen_pci_cfg_bus_ops {
- u32 bus_shift;
- struct pci_ops ops;
-};
-
-struct gen_pci_cfg_windows {
- struct resource res;
- struct resource *bus_range;
- void __iomem **win;
-
- struct gen_pci_cfg_bus_ops *ops;
-};
-
-struct gen_pci {
- struct pci_host_bridge host;
- struct gen_pci_cfg_windows cfg;
- struct list_head resources;
-};
-
-int pci_host_common_probe(struct platform_device *pdev,
- struct gen_pci *pci);
-
-#endif /* _PCI_HOST_COMMON_H */
diff --git a/drivers/pci/host/pci-host-generic.c b/drivers/pci/host/pci-host-generic.c
index e8aa78f..6eaceab 100644
--- a/drivers/pci/host/pci-host-generic.c
+++ b/drivers/pci/host/pci-host-generic.c
@@ -25,41 +25,12 @@
#include <linux/of_pci.h>
#include <linux/platform_device.h>

-#include "pci-host-common.h"
+#include "../ecam.h"

-static void __iomem *gen_pci_map_cfg_bus_cam(struct pci_bus *bus,
- unsigned int devfn,
- int where)
-{
- struct gen_pci *pci = bus->sysdata;
- resource_size_t idx = bus->number - pci->cfg.bus_range->start;
-
- return pci->cfg.win[idx] + ((devfn << 8) | where);
-}
-
-static struct gen_pci_cfg_bus_ops gen_pci_cfg_cam_bus_ops = {
+static struct pci_ecam_ops gen_pci_cfg_cam_bus_ops = {
.bus_shift = 16,
- .ops = {
- .map_bus = gen_pci_map_cfg_bus_cam,
- .read = pci_generic_config_read,
- .write = pci_generic_config_write,
- }
-};
-
-static void __iomem *gen_pci_map_cfg_bus_ecam(struct pci_bus *bus,
- unsigned int devfn,
- int where)
-{
- struct gen_pci *pci = bus->sysdata;
- resource_size_t idx = bus->number - pci->cfg.bus_range->start;
-
- return pci->cfg.win[idx] + ((devfn << 12) | where);
-}
-
-static struct gen_pci_cfg_bus_ops gen_pci_cfg_ecam_bus_ops = {
- .bus_shift = 20,
- .ops = {
- .map_bus = gen_pci_map_cfg_bus_ecam,
+ .pci_ops = {
+ .map_bus = pci_ecam_map_bus,
.read = pci_generic_config_read,
.write = pci_generic_config_write,
}
@@ -70,25 +41,22 @@ static const struct of_device_id gen_pci_of_match[] = {
.data = &gen_pci_cfg_cam_bus_ops },

{ .compatible = "pci-host-ecam-generic",
- .data = &gen_pci_cfg_ecam_bus_ops },
+ .data = &pci_generic_ecam_ops },

{ },
};
+
MODULE_DEVICE_TABLE(of, gen_pci_of_match);

static int gen_pci_probe(struct platform_device *pdev)
{
- struct device *dev = &pdev->dev;
const struct of_device_id *of_id;
- struct gen_pci *pci = devm_kzalloc(dev, sizeof(*pci), GFP_KERNEL);
-
- if (!pci)
- return -ENOMEM;
+ struct pci_ecam_ops *ops;

- of_id = of_match_node(gen_pci_of_match, dev->of_node);
- pci->cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data;
+ of_id = of_match_node(gen_pci_of_match, pdev->dev.of_node);
+ ops = (struct pci_ecam_ops *)of_id->data;

- return pci_host_common_probe(pdev, pci);
+ return pci_host_common_probe(pdev, ops);
}

static struct platform_driver gen_pci_driver = {
diff --git a/drivers/pci/host/pci-thunder-ecam.c b/drivers/pci/host/pci-thunder-ecam.c
index d71935cb..540d030 100644
--- a/drivers/pci/host/pci-thunder-ecam.c
+++ b/drivers/pci/host/pci-thunder-ecam.c
@@ -13,18 +13,7 @@
#include <linux/of.h>
#include <linux/platform_device.h>

-#include "pci-host-common.h"
-
-/* Mapping is standard ECAM */
-static void __iomem *thunder_ecam_map_bus(struct pci_bus *bus,
- unsigned int devfn,
- int where)
-{
- struct gen_pci *pci = bus->sysdata;
- resource_size_t idx = bus->number - pci->cfg.bus_range->start;
-
- return pci->cfg.win[idx] + ((devfn << 12) | where);
-}
+#include "../ecam.h"

static void set_val(u32 v, int where, int size, u32 *val)
{
@@ -99,7 +88,7 @@ static int handle_ea_bar(u32 e0, int bar, struct pci_bus *bus,
static int thunder_ecam_p2_config_read(struct pci_bus *bus, unsigned int devfn,
int where, int size, u32 *val)
{
- struct gen_pci *pci = bus->sysdata;
+ struct pci_config_window *cfg = bus->sysdata;
int where_a = where & ~3;
void __iomem *addr;
u32 node_bits;
@@ -129,7 +118,7 @@ static int thunder_ecam_p2_config_read(struct pci_bus *bus, unsigned int devfn,
* the config space access window. Since we are working with
* the high-order 32 bits, shift everything down by 32 bits.
*/
- node_bits = (pci->cfg.res.start >> 32) & (1 << 12);
+ node_bits = (cfg->res.start >> 32) & (1 << 12);

v |= node_bits;
set_val(v, where, size, val);
@@ -358,36 +347,24 @@ static int thunder_ecam_config_write(struct pci_bus *bus, unsigned int devfn,
return pci_generic_config_write(bus, devfn, where, size, val);
}

-static struct gen_pci_cfg_bus_ops thunder_ecam_bus_ops = {
+static struct pci_ecam_ops pci_thunder_ecam_ops = {
.bus_shift = 20,
- .ops = {
- .map_bus = thunder_ecam_map_bus,
+ .pci_ops = {
+ .map_bus = pci_ecam_map_bus,
.read = thunder_ecam_config_read,
.write = thunder_ecam_config_write,
}
};

static const struct of_device_id thunder_ecam_of_match[] = {
- { .compatible = "cavium,pci-host-thunder-ecam",
- .data = &thunder_ecam_bus_ops },
-
+ { .compatible = "cavium,pci-host-thunder-ecam" },
{ },
};
MODULE_DEVICE_TABLE(of, thunder_ecam_of_match);

static int thunder_ecam_probe(struct platform_device *pdev)
{
- struct device *dev = &pdev->dev;
- const struct of_device_id *of_id;
- struct gen_pci *pci = devm_kzalloc(dev, sizeof(*pci), GFP_KERNEL);
-
- if (!pci)
- return -ENOMEM;
-
- of_id = of_match_node(thunder_ecam_of_match, dev->of_node);
- pci->cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data;
-
- return pci_host_common_probe(pdev, pci);
+ return pci_host_common_probe(pdev, &pci_thunder_ecam_ops);
}

static struct platform_driver thunder_ecam_driver = {
diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c
index cabb92a..a7b61ce 100644
--- a/drivers/pci/host/pci-thunder-pem.c
+++ b/drivers/pci/host/pci-thunder-pem.c
@@ -20,34 +20,22 @@
#include <linux/of_pci.h>
#include <linux/platform_device.h>

-#include "pci-host-common.h"
+#include "../ecam.h"

#define PEM_CFG_WR 0x28
#define PEM_CFG_RD 0x30

struct thunder_pem_pci {
- struct gen_pci gen_pci;
u32 ea_entry[3];
void __iomem *pem_reg_base;
};

-static void __iomem *thunder_pem_map_bus(struct pci_bus *bus,
- unsigned int devfn, int where)
-{
- struct gen_pci *pci = bus->sysdata;
- resource_size_t idx = bus->number - pci->cfg.bus_range->start;
-
- return pci->cfg.win[idx] + ((devfn << 16) | where);
-}
-
static int thunder_pem_bridge_read(struct pci_bus *bus, unsigned int devfn,
int where, int size, u32 *val)
{
u64 read_val;
- struct thunder_pem_pci *pem_pci;
- struct gen_pci *pci = bus->sysdata;
-
- pem_pci = container_of(pci, struct thunder_pem_pci, gen_pci);
+ struct pci_config_window *cfg = bus->sysdata;
+ struct thunder_pem_pci *pem_pci = (struct thunder_pem_pci *)cfg->priv;

if (devfn != 0 || where >= 2048) {
*val = ~0;
@@ -132,17 +120,17 @@ static int thunder_pem_bridge_read(struct pci_bus *bus, unsigned int devfn,
static int thunder_pem_config_read(struct pci_bus *bus, unsigned int devfn,
int where, int size, u32 *val)
{
- struct gen_pci *pci = bus->sysdata;
+ struct pci_config_window *cfg = bus->sysdata;

- if (bus->number < pci->cfg.bus_range->start ||
- bus->number > pci->cfg.bus_range->end)
+ if (bus->number < cfg->busr.start ||
+ bus->number > cfg->busr.end)
return PCIBIOS_DEVICE_NOT_FOUND;

/*
* The first device on the bus is the PEM PCIe bridge.
* Special case its config access.
*/
- if (bus->number == pci->cfg.bus_range->start)
+ if (bus->number == cfg->busr.start)
return thunder_pem_bridge_read(bus, devfn, where, size, val);

return pci_generic_config_read(bus, devfn, where, size, val);
@@ -187,12 +175,11 @@ static u32 thunder_pem_bridge_w1c_bits(int where)
static int thunder_pem_bridge_write(struct pci_bus *bus, unsigned int devfn,
int where, int size, u32 val)
{
- struct gen_pci *pci = bus->sysdata;
- struct thunder_pem_pci *pem_pci;
+ struct pci_config_window *cfg = bus->sysdata;
+ struct thunder_pem_pci *pem_pci = (struct thunder_pem_pci *)cfg->priv;
u64 write_val, read_val;
u32 mask = 0;

- pem_pci = container_of(pci, struct thunder_pem_pci, gen_pci);

if (devfn != 0 || where >= 2048)
return PCIBIOS_DEVICE_NOT_FOUND;
@@ -256,53 +243,38 @@ static int thunder_pem_bridge_write(struct pci_bus *bus, unsigned int devfn,
static int thunder_pem_config_write(struct pci_bus *bus, unsigned int devfn,
int where, int size, u32 val)
{
- struct gen_pci *pci = bus->sysdata;
+ struct pci_config_window *cfg = bus->sysdata;

- if (bus->number < pci->cfg.bus_range->start ||
- bus->number > pci->cfg.bus_range->end)
+ if (bus->number < cfg->busr.start ||
+ bus->number > cfg->busr.end)
return PCIBIOS_DEVICE_NOT_FOUND;
/*
* The first device on the bus is the PEM PCIe bridge.
* Special case its config access.
*/
- if (bus->number == pci->cfg.bus_range->start)
+ if (bus->number == cfg->busr.start)
return thunder_pem_bridge_write(bus, devfn, where, size, val);


return pci_generic_config_write(bus, devfn, where, size, val);
}

-static struct gen_pci_cfg_bus_ops thunder_pem_bus_ops = {
- .bus_shift = 24,
- .ops = {
- .map_bus = thunder_pem_map_bus,
- .read = thunder_pem_config_read,
- .write = thunder_pem_config_write,
- }
-};
-
-static const struct of_device_id thunder_pem_of_match[] = {
- { .compatible = "cavium,pci-host-thunder-pem",
- .data = &thunder_pem_bus_ops },
-
- { },
-};
-MODULE_DEVICE_TABLE(of, thunder_pem_of_match);
-
-static int thunder_pem_probe(struct platform_device *pdev)
+static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg)
{
- struct device *dev = &pdev->dev;
- const struct of_device_id *of_id;
resource_size_t bar4_start;
struct resource *res_pem;
struct thunder_pem_pci *pem_pci;
+ struct platform_device *pdev;
+
+ /* Only OF support for now */
+ if (!dev->of_node)
+ return -EINVAL;

pem_pci = devm_kzalloc(dev, sizeof(*pem_pci), GFP_KERNEL);
if (!pem_pci)
return -ENOMEM;

- of_id = of_match_node(thunder_pem_of_match, dev->of_node);
- pem_pci->gen_pci.cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data;
+ pdev = to_platform_device(dev);

/*
* The second register range is the PEM bridge to the PCIe
@@ -330,7 +302,29 @@ static int thunder_pem_probe(struct platform_device *pdev)
pem_pci->ea_entry[1] = (u32)(res_pem->end - bar4_start) & ~3u;
pem_pci->ea_entry[2] = (u32)(bar4_start >> 32);

- return pci_host_common_probe(pdev, &pem_pci->gen_pci);
+ cfg->priv = pem_pci;
+ return 0;
+}
+
+static struct pci_ecam_ops pci_thunder_pem_ops = {
+ .bus_shift = 24,
+ .init = thunder_pem_init,
+ .pci_ops = {
+ .map_bus = pci_ecam_map_bus,
+ .read = thunder_pem_config_read,
+ .write = thunder_pem_config_write,
+ }
+};
+
+static const struct of_device_id thunder_pem_of_match[] = {
+ { .compatible = "cavium,pci-host-thunder-pem" },
+ { },
+};
+MODULE_DEVICE_TABLE(of, thunder_pem_of_match);
+
+static int thunder_pem_probe(struct platform_device *pdev)
+{
+ return pci_host_common_probe(pdev, &pci_thunder_pem_ops);
}

static struct platform_driver thunder_pem_driver = {
--
1.9.1
Gabriele Paoloni
2016-05-11 10:50:02 UTC
Permalink
Hi Tomasz
-----Original Message-----
Sent: 10 May 2016 16:20
Subject: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
From the functionality point of view this series may be split into the
1. New ECAM API and update for users of the pci-host-common API
2. Necessary fixes as the preparation for using driver on ARM64.
3. Use new MCFG interface and implement generic ACPI based PCI host
controller driver.
4. Enable above driver on ARM64
This has been tested on Cavium ThunderX server. Any help in reviewing
and
testing is very appreciated.
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it
decided to drop quirk handling?

I think it is important to have this in place to accommodate different
vendors. If the intention is to keep this patchset "clean" maybe
we can add it as a separate patch on top later on...

What’s your view?

Thanks

Gab
- changes for ACPI companion and domain number assignment approach
- implement arch pcibios_{add|remove}_bus and call
acpi_pci_{add|remove}_bus from there
- cleanups around nomenclature
- use resources oriented API for ECAM
- fix for based address calculation before mapping ECAM region
- remove useless lock for MCFG lookup
- move MCFG stuff to separated file pci_mcfg.c
- drop MCFG entries caching
- rebase against 4.6-rc7
v5 -> v6
- drop idea of x86 MMCONFIG code refactoring
https://lkml.org/lkml/2016/4/11/907
git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
- integrate Sinan's fix for releasing IO resources, see patch [06/13]
- added ACPI support for ThunderX ECAM and PEM drivers
- rebase against 4.6-rc2
v4 -> v5
- drop MCFG refactoring group patches 1-6 from series v4 and integrate
Jayachandran's patch
https://patchwork.ozlabs.org/patch/575525/
- rewrite PCI legacy IRQs allocation
- squash two patches 11 and 12 from series v4, fixed bisection issue
- changelog improvements
- rebase against 4.5-rc3
v3 -> v4
- drop Jiang's fix
http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
- add Lorenzo's fix patch 19/24
- ACPI PCI bus domain number assigning cleanup
- change resource management, we now claim and reassign resources
- improvements for applying quirks
- drop Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html
dependency
- rebase against 4.5-rc1
v2 -> v3
- fix legacy IRQ assigning and IO ports registration
- remove reference to arch specific companion device for ia64
- move ACPI PCI host controller driver to pci_root.c
- drop generic domain assignment for x86 and ia64 as I am not
able to run all necessary test variants
- drop patch which cleaned legacy IRQ assignment since it belongs to
https://patchwork.ozlabs.org/patch/557504/
- extend MCFG quirk code
- rebase against 4.4
v1 -> v2
- move non-arch specific piece of code to dirver/acpi/ directory
- fix IO resource handling
- introduce PCI config accessors quirks matching
- moved ACPI_COMPANION_SET to generic code
v1 - https://lkml.org/lkml/2015/10/27/504
v2 - https://lkml.org/lkml/2015/12/16/246
v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html
v4 - https://lkml.org/lkml/2016/2/4/646
v5 - https://lkml.org/lkml/2016/2/16/426
v6 - https://lkml.org/lkml/2016/4/15/594
PCI: Provide common functions for ECAM mapping
PCI: generic, thunder: update to use generic ECAM API
pci, of: Move the PCI I/O space management to PCI core code.
pci: Add new function to unmap IO resources.
acpi, pci: Support IO resources when parsing PCI host bridge
resources.
pci, acpi: Provide a way to assign bus domain number.
pci, acpi: Handle ACPI companion assignment.
pci, acpi: Support for ACPI based generic PCI host controller
arm64, pci, acpi: ACPI support for legacy IRQs parsing and
consolidation with DT code.
arm64, pci, acpi: Provide ACPI-specific prerequisites for PCI bus
enumeration.
arm64, pci, acpi: Start using ACPI based PCI host controller driver
for ARM64.
arch/arm64/Kconfig | 1 +
arch/arm64/kernel/pci.c | 34 +++-----
drivers/acpi/Kconfig | 8 ++
drivers/acpi/Makefile | 1 +
drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++
drivers/acpi/pci_root.c | 33 ++++++++
drivers/acpi/pci_root_generic.c | 149
+++++++++++++++++++++++++++++++++
drivers/of/address.c | 116 +-------------------------
drivers/pci/Kconfig | 3 +
drivers/pci/Makefile | 2 +
drivers/pci/ecam.c | 161
++++++++++++++++++++++++++++++++++++
drivers/pci/ecam.h | 72 ++++++++++++++++
drivers/pci/host/Kconfig | 1 +
drivers/pci/host/pci-host-common.c | 114 +++++++++++--------------
drivers/pci/host/pci-host-common.h | 47 -----------
drivers/pci/host/pci-host-generic.c | 52 +++---------
drivers/pci/host/pci-thunder-ecam.c | 39 ++-------
drivers/pci/host/pci-thunder-pem.c | 92 ++++++++++-----------
drivers/pci/pci.c | 150
++++++++++++++++++++++++++++++++-
drivers/pci/probe.c | 2 +
include/linux/of_address.h | 9 --
include/linux/pci-acpi.h | 14 ++++
include/linux/pci.h | 11 ++-
23 files changed, 823 insertions(+), 385 deletions(-)
create mode 100644 drivers/acpi/pci_mcfg.c
create mode 100644 drivers/acpi/pci_root_generic.c
create mode 100644 drivers/pci/ecam.c
create mode 100644 drivers/pci/ecam.h
delete mode 100644 drive
Tomasz Nowicki
2016-05-11 11:10:02 UTC
Permalink
Hi Gabriele,
Post by Gabriele Paoloni
Hi Tomasz
-----Original Message-----
Sent: 10 May 2016 16:20
Subject: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
From the functionality point of view this series may be split into the
1. New ECAM API and update for users of the pci-host-common API
2. Necessary fixes as the preparation for using driver on ARM64.
3. Use new MCFG interface and implement generic ACPI based PCI host
controller driver.
4. Enable above driver on ARM64
This has been tested on Cavium ThunderX server. Any help in reviewing
and
testing is very appreciated.
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it
decided to drop quirk handling?
I had such requests in previous series.
Post by Gabriele Paoloni
I think it is important to have this in place to accommodate different
vendors. If the intention is to keep this patchset "clean" maybe
we can add it as a separate patch on top later on...
What’s your view?
Yes, keeping these things separated should help in review. Obviously I
agree that we need quirk handling but currently there is no
implementation which we all agree upon. For the test, you can use quirk
handling approach from the previous series until we sort out final solution.

Thanks,
Tomasz
Gabriele Paoloni
2016-05-11 13:00:04 UTC
Permalink
Hi Tomasz
-----Original Message-----
Sent: 11 May 2016 12:08
Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
Hi Gabriele,
Post by Gabriele Paoloni
Hi Tomasz
-----Original Message-----
Sent: 10 May 2016 16:20
Subject: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
From the functionality point of view this series may be split into
the
Post by Gabriele Paoloni
1. New ECAM API and update for users of the pci-host-common API
2. Necessary fixes as the preparation for using driver on ARM64.
3. Use new MCFG interface and implement generic ACPI based PCI host
controller driver.
4. Enable above driver on ARM64
This has been tested on Cavium ThunderX server. Any help in
reviewing
Post by Gabriele Paoloni
and
testing is very appreciated.
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it
decided to drop quirk handling?
I had such requests in previous series.
Post by Gabriele Paoloni
I think it is important to have this in place to accommodate
different
Post by Gabriele Paoloni
vendors. If the intention is to keep this patchset "clean" maybe
we can add it as a separate patch on top later on...
What's your view?
Yes, keeping these things separated should help in review. Obviously I
agree that we need quirk handling but currently there is no
implementation which we all agree upon. For the test, you can use quirk
handling approach from the previous series until we sort out final solution.
Great

This explains perfectly

We will apply previous series quirks and try to test the patchset
from Huawei side

Many Thanks

Gab
Thanks,
Tomasz
Jon Masters
2016-05-20 04:50:02 UTC
Permalink
Hi Tomasz, all,
Post by Tomasz Nowicki
Post by Gabriele Paoloni
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it
decided to drop quirk handling?
I had such requests in previous series.
A quick note on quirk handling. This, I believe, applies post-merge of
the base infrastructure, which I realize will likely not have quirks.

We've some "gen1" ARMv8 server platforms where we end up doing quirks
(for things like forcing 32-bit config space accessors and the like) due
to people repurposing existing embedded PCIe IP blocks or using them for
the first time (especially in servers), and those being involved in the
design not necessarily seeing this problem ahead of time, or not
realizing that it would be an issue for servers. In the early days of
ARM server designs 3-4 years ago, many of us had never really played
with ECAM or realized how modern topologies are built.

Anyway. We missed this one in our SBSA requirements. They say (words to
the effect of) "thou shalt do PCIe the way it is done on servers" but
they aren't prescriptive, and they don't tell people how that actually
is in reality. That is being fixed. A lot of things are happening behind
the scenes - especially with third party IP block providers (all of whom
myself and others are speaking with directly about this) - to ensure
that the next wave of designs won't repeat these mistakes. We don't have
a time machine, but we can contain this from becoming an ongoing mess
for upstream, and we will do so. It won't be a zoo.

Various proposals have arisen for how to handle quirks in the longer
term, including elaborate frameworks and tables to describe them
generically. I would like to caution against such approaches, especially
in the case that they deviate from practice on x86, or prior to being
standardized fully with other Operating System vendors. I don't expect
there to be too many more than the existing initial set of quirks we
have seen posted. A number of "future" server SoCs have already been
fixed prior to silicon, and new design starts are being warned not to
make this a problem for us to have to clean up later.

So, I would like to suggest that the eventual framework mirror the
existing approach on x86 systems (matching DMI, etc.) and not be made
into some kind of generic, utopia. This is a case where we want there to
be pain involved (and upstream patches required) when people screw up,
so that they have a level of pain in response to ever making this
mistake in the future. If we try to create too grand a generic scheme
and make it too easy to handle this kind of situation beyond the small
number of existing offenders, we undermine efforts to force vendors to
ensure that their IP blocks are compliant going forward.

Side note: if you're a third party IP vendor and we didn't already speak
about this one, drop me a line, and let's collaborate also on your test
frameworks to make sure you're covered as well.

Jon.
--
Computer Architect | Sent from my Fedora powered laptop
Ard Biesheuvel
2016-05-20 07:40:02 UTC
Permalink
Post by Jon Masters
Hi Tomasz, all,
Post by Tomasz Nowicki
Post by Gabriele Paoloni
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it
decided to drop quirk handling?
I had such requests in previous series.
A quick note on quirk handling. This, I believe, applies post-merge of
the base infrastructure, which I realize will likely not have quirks.
We've some "gen1" ARMv8 server platforms where we end up doing quirks
(for things like forcing 32-bit config space accessors and the like) due
to people repurposing existing embedded PCIe IP blocks or using them for
the first time (especially in servers), and those being involved in the
design not necessarily seeing this problem ahead of time, or not
realizing that it would be an issue for servers. In the early days of
ARM server designs 3-4 years ago, many of us had never really played
with ECAM or realized how modern topologies are built.
Anyway. We missed this one in our SBSA requirements. They say (words to
the effect of) "thou shalt do PCIe the way it is done on servers" but
they aren't prescriptive, and they don't tell people how that actually
is in reality. That is being fixed. A lot of things are happening behind
the scenes - especially with third party IP block providers (all of whom
myself and others are speaking with directly about this) - to ensure
that the next wave of designs won't repeat these mistakes. We don't have
a time machine, but we can contain this from becoming an ongoing mess
for upstream, and we will do so. It won't be a zoo.
Various proposals have arisen for how to handle quirks in the longer
term, including elaborate frameworks and tables to describe them
generically. I would like to caution against such approaches, especially
in the case that they deviate from practice on x86, or prior to being
standardized fully with other Operating System vendors. I don't expect
there to be too many more than the existing initial set of quirks we
have seen posted. A number of "future" server SoCs have already been
fixed prior to silicon, and new design starts are being warned not to
make this a problem for us to have to clean up later.
So, I would like to suggest that the eventual framework mirror the
existing approach on x86 systems (matching DMI, etc.) and not be made
into some kind of generic, utopia. This is a case where we want there to
be pain involved (and upstream patches required) when people screw up,
so that they have a level of pain in response to ever making this
mistake in the future. If we try to create too grand a generic scheme
and make it too easy to handle this kind of situation beyond the small
number of existing offenders, we undermine efforts to force vendors to
ensure that their IP blocks are compliant going forward.
I understand that there is a desire from the RedHat side to mimic x86
as closely as possible, but I never saw any technical justification
for that. DMI contains strings that are visible to userland, and you
effectively lock those down to certain values just so that the kernel
can distinguish a broken PCIe root complex from a working one. Linux
on x86 had no choice, since the overwhelming majority of existing
hardware misrepresented itself as generic, and DMI was the only thing
available to actually distinguish these broken implementations from
one another. This does not mean we should allow and/or encourage this
first gen hardware to misrepresent non-compliant hardware as compliant
as well.

Since you are talking to all the people involved, how about you
convince them to put something in the ACPI tables that allows the
kernel to distinguish those non-standard PCIe implementations from
hardware that is really generic? This way, we can sidestep the quirks
debate entirely, since it will simply be a different device as far as
the kernel is concerned. This is no worse than a quirk from a
practical point of view, since an older OS will be equally unable to
run on newer hardware, but it is arguably more true to the standards
compliance you tend to preach about, especially since this small pool
of third party IP could potentially be identified directly rather than
based on some divination of the SoC we may or may not be running on. I
am also convinced that adding support for an additional HID() to the
ACPI ECAM driver with some special config space handling wired in is
an easier sell upstream than making the same ugly mess x86 has had to
make because they did not have any choice to begin with.

If we do need a quirks handling mechanism, I still don't see how the
x86 situation extrapolates to ARM. ACPI offers plenty of ways for a
SoC vendor to identify the make and particular revision, and quirks
could be keyed off of that.
--
Ard.
Jon Masters
2016-05-20 08:10:01 UTC
Permalink
Hi Ard,
Post by Ard Biesheuvel
Post by Jon Masters
Hi Tomasz, all,
Post by Tomasz Nowicki
Post by Gabriele Paoloni
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it
decided to drop quirk handling?
I had such requests in previous series.
A quick note on quirk handling. This, I believe, applies post-merge of
the base infrastructure, which I realize will likely not have quirks.
We've some "gen1" ARMv8 server platforms where we end up doing quirks
(for things like forcing 32-bit config space accessors and the like) due
to people repurposing existing embedded PCIe IP blocks or using them for
the first time (especially in servers), and those being involved in the
design not necessarily seeing this problem ahead of time, or not
realizing that it would be an issue for servers. In the early days of
ARM server designs 3-4 years ago, many of us had never really played
with ECAM or realized how modern topologies are built.
Anyway. We missed this one in our SBSA requirements. They say (words to
the effect of) "thou shalt do PCIe the way it is done on servers" but
they aren't prescriptive, and they don't tell people how that actually
is in reality. That is being fixed. A lot of things are happening behind
the scenes - especially with third party IP block providers (all of whom
myself and others are speaking with directly about this) - to ensure
that the next wave of designs won't repeat these mistakes. We don't have
a time machine, but we can contain this from becoming an ongoing mess
for upstream, and we will do so. It won't be a zoo.
Various proposals have arisen for how to handle quirks in the longer
term, including elaborate frameworks and tables to describe them
generically. I would like to caution against such approaches, especially
in the case that they deviate from practice on x86, or prior to being
standardized fully with other Operating System vendors. I don't expect
there to be too many more than the existing initial set of quirks we
have seen posted. A number of "future" server SoCs have already been
fixed prior to silicon, and new design starts are being warned not to
make this a problem for us to have to clean up later.
So, I would like to suggest that the eventual framework mirror the
existing approach on x86 systems (matching DMI, etc.) and not be made
into some kind of generic, utopia. This is a case where we want there to
be pain involved (and upstream patches required) when people screw up,
so that they have a level of pain in response to ever making this
mistake in the future. If we try to create too grand a generic scheme
and make it too easy to handle this kind of situation beyond the small
number of existing offenders, we undermine efforts to force vendors to
ensure that their IP blocks are compliant going forward.
I understand that there is a desire from the RedHat side to mimic x86
as closely as possible, but I never saw any technical justification
for that.
Understood. My own motivation is always to make the experience as
familiar as possible, both for end users, as well as for ODMs and the
entire ecosystem. There are very many ODMs currently working on v8
server designs and they're already expecting this to be "just like x88".
Intentionally. But as to the specifics of using DMI...
Post by Ard Biesheuvel
DMI contains strings that are visible to userland, and you
effectively lock those down to certain values just so that the kernel
can distinguish a broken PCIe root complex from a working one. Linux
on x86 had no choice, since the overwhelming majority of existing
hardware misrepresented itself as generic, and DMI was the only thing
available to actually distinguish these broken implementations from
one another. This does not mean we should allow and/or encourage this
first gen hardware to misrepresent non-compliant hardware as compliant
as well.
That's a very reasonable argument. I don't disagree that it would be
nice to have nicer ways to distinguish the non-compliant IP than
treating the whole platform with an ASCII matching sledgehammer.
Post by Ard Biesheuvel
Since you are talking to all the people involved, how about you
convince them to put something in the ACPI tables that allows the
kernel to distinguish those non-standard PCIe implementations from
hardware that is really generic?
I'm open to this *BUT* it has to be something that will be adopted
beyond Linux. I have reached out to some non-Linux folks about this. If
there's buy-in, and if there's agreement to go standardize it through
the ASWG, then we should do so. What we should not do is treat ARM as
special in a way that the others aren't involved with. I'll admit DMI
ended up part of the SBBR in part because I wrote that piece in with the
assumption that exactly the same matches as on x86 would happen.
Post by Ard Biesheuvel
This way, we can sidestep the quirks
debate entirely, since it will simply be a different device as far as
the kernel is concerned. This is no worse than a quirk from a
practical point of view, since an older OS will be equally unable to
run on newer hardware, but it is arguably more true to the standards
compliance you tend to preach about, especially since this small pool
of third party IP could potentially be identified directly rather than
based on some divination of the SoC we may or may not be running on. I
am also convinced that adding support for an additional HID() to the
ACPI ECAM driver with some special config space handling wired in is
an easier sell upstream than making the same ugly mess x86 has had to
make because they did not have any choice to begin with.
Again, open to it. I just don't want to do something that's Linux
specific. So it'll take time. It would be awesome if an interim quirk
solution existed that got platforms that are shipping in production
(e.g. HP Moonshot) actually booting upstream kernels this year. We
/really/ want F25 to be able to run on these without needing to carry an
out-of-tree quirk patch or just not support them. That's much worse.
Post by Ard Biesheuvel
If we do need a quirks handling mechanism, I still don't see how the
x86 situation extrapolates to ARM. ACPI offers plenty of ways for a
SoC vendor to identify the make and particular revision, and quirks
could be keyed off of that.
Fair enough. Is there any traction for an interim solution for these
initial platforms do you think? If we wait to add a new table it's
probably going to be the end of the year before we get this done.

Jon.
--
Computer Architect | Sent from my Fedora powered laptop
Ard Biesheuvel
2016-05-20 08:30:02 UTC
Permalink
Post by Jon Masters
Hi Ard,
Post by Ard Biesheuvel
Post by Jon Masters
Hi Tomasz, all,
Post by Tomasz Nowicki
Post by Gabriele Paoloni
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it
decided to drop quirk handling?
I had such requests in previous series.
A quick note on quirk handling. This, I believe, applies post-merge of
the base infrastructure, which I realize will likely not have quirks.
We've some "gen1" ARMv8 server platforms where we end up doing quirks
(for things like forcing 32-bit config space accessors and the like) due
to people repurposing existing embedded PCIe IP blocks or using them for
the first time (especially in servers), and those being involved in the
design not necessarily seeing this problem ahead of time, or not
realizing that it would be an issue for servers. In the early days of
ARM server designs 3-4 years ago, many of us had never really played
with ECAM or realized how modern topologies are built.
Anyway. We missed this one in our SBSA requirements. They say (words to
the effect of) "thou shalt do PCIe the way it is done on servers" but
they aren't prescriptive, and they don't tell people how that actually
is in reality. That is being fixed. A lot of things are happening behind
the scenes - especially with third party IP block providers (all of whom
myself and others are speaking with directly about this) - to ensure
that the next wave of designs won't repeat these mistakes. We don't have
a time machine, but we can contain this from becoming an ongoing mess
for upstream, and we will do so. It won't be a zoo.
Various proposals have arisen for how to handle quirks in the longer
term, including elaborate frameworks and tables to describe them
generically. I would like to caution against such approaches, especially
in the case that they deviate from practice on x86, or prior to being
standardized fully with other Operating System vendors. I don't expect
there to be too many more than the existing initial set of quirks we
have seen posted. A number of "future" server SoCs have already been
fixed prior to silicon, and new design starts are being warned not to
make this a problem for us to have to clean up later.
So, I would like to suggest that the eventual framework mirror the
existing approach on x86 systems (matching DMI, etc.) and not be made
into some kind of generic, utopia. This is a case where we want there to
be pain involved (and upstream patches required) when people screw up,
so that they have a level of pain in response to ever making this
mistake in the future. If we try to create too grand a generic scheme
and make it too easy to handle this kind of situation beyond the small
number of existing offenders, we undermine efforts to force vendors to
ensure that their IP blocks are compliant going forward.
I understand that there is a desire from the RedHat side to mimic x86
as closely as possible, but I never saw any technical justification
for that.
Understood. My own motivation is always to make the experience as
familiar as possible, both for end users, as well as for ODMs and the
entire ecosystem. There are very many ODMs currently working on v8
server designs and they're already expecting this to be "just like x88".
Intentionally. But as to the specifics of using DMI...
Post by Ard Biesheuvel
DMI contains strings that are visible to userland, and you
effectively lock those down to certain values just so that the kernel
can distinguish a broken PCIe root complex from a working one. Linux
on x86 had no choice, since the overwhelming majority of existing
hardware misrepresented itself as generic, and DMI was the only thing
available to actually distinguish these broken implementations from
one another. This does not mean we should allow and/or encourage this
first gen hardware to misrepresent non-compliant hardware as compliant
as well.
That's a very reasonable argument. I don't disagree that it would be
nice to have nicer ways to distinguish the non-compliant IP than
treating the whole platform with an ASCII matching sledgehammer.
Post by Ard Biesheuvel
Since you are talking to all the people involved, how about you
convince them to put something in the ACPI tables that allows the
kernel to distinguish those non-standard PCIe implementations from
hardware that is really generic?
I'm open to this *BUT* it has to be something that will be adopted
beyond Linux. I have reached out to some non-Linux folks about this. If
there's buy-in, and if there's agreement to go standardize it through
the ASWG, then we should do so. What we should not do is treat ARM as
special in a way that the others aren't involved with. I'll admit DMI
ended up part of the SBBR in part because I wrote that piece in with the
assumption that exactly the same matches as on x86 would happen.
Is the PCIe root complex so special that you cannot simply describe an
implementation that is not PNP0408 compatible as something else, under
its own unique HID? If everybody is onboard with using ACPI, how is
this any different from describing other parts of the platform
topology? Even if the SBSA mandates generic PCI, they already deviated
from that when they built the hardware, so pretending that it is a
PNP0408 with quirks really does not buy us anything.
Post by Jon Masters
Post by Ard Biesheuvel
This way, we can sidestep the quirks
debate entirely, since it will simply be a different device as far as
the kernel is concerned. This is no worse than a quirk from a
practical point of view, since an older OS will be equally unable to
run on newer hardware, but it is arguably more true to the standards
compliance you tend to preach about, especially since this small pool
of third party IP could potentially be identified directly rather than
based on some divination of the SoC we may or may not be running on. I
am also convinced that adding support for an additional HID() to the
ACPI ECAM driver with some special config space handling wired in is
an easier sell upstream than making the same ugly mess x86 has had to
make because they did not have any choice to begin with.
Again, open to it. I just don't want to do something that's Linux
specific. So it'll take time. It would be awesome if an interim quirk
solution existed that got platforms that are shipping in production
(e.g. HP Moonshot) actually booting upstream kernels this year. We
/really/ want F25 to be able to run on these without needing to carry an
out-of-tree quirk patch or just not support them. That's much worse.
My whole point is that we don't need quirks in the first place if
non-compliant devices are not being misrepresented as compliant ones.
This is fine for other platform devices, i.e., SATA, network, so
again, why is PCIe so special that we *must* use a generic ID + quirks
rather than a specific ID?
Post by Jon Masters
Post by Ard Biesheuvel
If we do need a quirks handling mechanism, I still don't see how the
x86 situation extrapolates to ARM. ACPI offers plenty of ways for a
SoC vendor to identify the make and particular revision, and quirks
could be keyed off of that.
Fair enough. Is there any traction for an interim solution for these
initial platforms do you think? If we wait to add a new table it's
probably going to be the end of the year before we get this done.
The 'interim solution' is to come to terms with the fact that these
initial platforms are not SBSA compliant, contain a PCIe root complex
that is not PNP0408 but can be identified by its own HID, and we make
the software work with that. This means our message from the beginning
is that, yes, you can have non-compliant hardware and the burden is on
you to get it supported upstream, and no, you don't get to hang out
with the cool SBSA kids if you decide to go that route
Gabriele Paoloni
2016-05-20 08:50:01 UTC
Permalink
Hi Ard
-----Original Message-----
Sent: 20 May 2016 09:29
To: Jon Masters
Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
Post by Jon Masters
Hi Ard,
Post by Ard Biesheuvel
Post by Jon Masters
Hi Tomasz, all,
Post by Tomasz Nowicki
Post by Gabriele Paoloni
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it
decided to drop quirk handling?
I had such requests in previous series.
A quick note on quirk handling. This, I believe, applies post-merge
of
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
the base infrastructure, which I realize will likely not have
quirks.
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
We've some "gen1" ARMv8 server platforms where we end up doing
quirks
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
(for things like forcing 32-bit config space accessors and the
like) due
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
to people repurposing existing embedded PCIe IP blocks or using
them for
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
the first time (especially in servers), and those being involved in
the
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
design not necessarily seeing this problem ahead of time, or not
realizing that it would be an issue for servers. In the early days
of
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
ARM server designs 3-4 years ago, many of us had never really
played
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
with ECAM or realized how modern topologies are built.
Anyway. We missed this one in our SBSA requirements. They say
(words to
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
the effect of) "thou shalt do PCIe the way it is done on servers"
but
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
they aren't prescriptive, and they don't tell people how that
actually
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
is in reality. That is being fixed. A lot of things are happening
behind
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
the scenes - especially with third party IP block providers (all of
whom
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
myself and others are speaking with directly about this) - to
ensure
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
that the next wave of designs won't repeat these mistakes. We don't
have
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
a time machine, but we can contain this from becoming an ongoing
mess
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
for upstream, and we will do so. It won't be a zoo.
Various proposals have arisen for how to handle quirks in the
longer
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
term, including elaborate frameworks and tables to describe them
generically. I would like to caution against such approaches,
especially
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
in the case that they deviate from practice on x86, or prior to
being
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
standardized fully with other Operating System vendors. I don't
expect
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
there to be too many more than the existing initial set of quirks
we
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
have seen posted. A number of "future" server SoCs have already
been
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
fixed prior to silicon, and new design starts are being warned not
to
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
make this a problem for us to have to clean up later.
So, I would like to suggest that the eventual framework mirror the
existing approach on x86 systems (matching DMI, etc.) and not be
made
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
into some kind of generic, utopia. This is a case where we want
there to
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
be pain involved (and upstream patches required) when people screw
up,
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
so that they have a level of pain in response to ever making this
mistake in the future. If we try to create too grand a generic
scheme
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
and make it too easy to handle this kind of situation beyond the
small
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
number of existing offenders, we undermine efforts to force vendors
to
Post by Jon Masters
Post by Ard Biesheuvel
Post by Jon Masters
ensure that their IP blocks are compliant going forward.
I understand that there is a desire from the RedHat side to mimic
x86
Post by Jon Masters
Post by Ard Biesheuvel
as closely as possible, but I never saw any technical justification
for that.
Understood. My own motivation is always to make the experience as
familiar as possible, both for end users, as well as for ODMs and the
entire ecosystem. There are very many ODMs currently working on v8
server designs and they're already expecting this to be "just like
x88".
Post by Jon Masters
Intentionally. But as to the specifics of using DMI...
Post by Ard Biesheuvel
DMI contains strings that are visible to userland, and you
effectively lock those down to certain values just so that the
kernel
Post by Jon Masters
Post by Ard Biesheuvel
can distinguish a broken PCIe root complex from a working one. Linux
on x86 had no choice, since the overwhelming majority of existing
hardware misrepresented itself as generic, and DMI was the only
thing
Post by Jon Masters
Post by Ard Biesheuvel
available to actually distinguish these broken implementations from
one another. This does not mean we should allow and/or encourage
this
Post by Jon Masters
Post by Ard Biesheuvel
first gen hardware to misrepresent non-compliant hardware as
compliant
Post by Jon Masters
Post by Ard Biesheuvel
as well.
That's a very reasonable argument. I don't disagree that it would be
nice to have nicer ways to distinguish the non-compliant IP than
treating the whole platform with an ASCII matching sledgehammer.
Post by Ard Biesheuvel
Since you are talking to all the people involved, how about you
convince them to put something in the ACPI tables that allows the
kernel to distinguish those non-standard PCIe implementations from
hardware that is really generic?
I'm open to this *BUT* it has to be something that will be adopted
beyond Linux. I have reached out to some non-Linux folks about this.
If
Post by Jon Masters
there's buy-in, and if there's agreement to go standardize it through
the ASWG, then we should do so. What we should not do is treat ARM as
special in a way that the others aren't involved with. I'll admit DMI
ended up part of the SBBR in part because I wrote that piece in with
the
Post by Jon Masters
assumption that exactly the same matches as on x86 would happen.
Is the PCIe root complex so special that you cannot simply describe an
implementation that is not PNP0408 compatible as something else, under
its own unique HID? If everybody is onboard with using ACPI, how is
this any different from describing other parts of the platform
topology? Even if the SBSA mandates generic PCI, they already deviated
from that when they built the hardware, so pretending that it is a
PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow each
vendor to come up with his own code and it would be much more effort
for the PCI maintainer to rework the PCI framework to accommodate X86
and "all" ARM64 Host Controllers...

I guess this approach is too risky and we want to avoid this. Through
standardization we can more easily maintain the code and scale it to
multiple SoCs...

So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
a bit more explanation...

Thanks

Gab
Post by Jon Masters
Post by Ard Biesheuvel
This way, we can sidestep the quirks
debate entirely, since it will simply be a different device as far
as
Post by Jon Masters
Post by Ard Biesheuvel
the kernel is concerned. This is no worse than a quirk from a
practical point of view, since an older OS will be equally unable to
run on newer hardware, but it is arguably more true to the standards
compliance you tend to preach about, especially since this small
pool
Post by Jon Masters
Post by Ard Biesheuvel
of third party IP could potentially be identified directly rather
than
Post by Jon Masters
Post by Ard Biesheuvel
based on some divination of the SoC we may or may not be running on.
I
Post by Jon Masters
Post by Ard Biesheuvel
am also convinced that adding support for an additional HID() to the
ACPI ECAM driver with some special config space handling wired in is
an easier sell upstream than making the same ugly mess x86 has had
to
Post by Jon Masters
Post by Ard Biesheuvel
make because they did not have any choice to begin with.
Again, open to it. I just don't want to do something that's Linux
specific. So it'll take time. It would be awesome if an interim quirk
solution existed that got platforms that are shipping in production
(e.g. HP Moonshot) actually booting upstream kernels this year. We
/really/ want F25 to be able to run on these without needing to carry
an
Post by Jon Masters
out-of-tree quirk patch or just not support them. That's much worse.
My whole point is that we don't need quirks in the first place if
non-compliant devices are not being misrepresented as compliant ones.
This is fine for other platform devices, i.e., SATA, network, so
again, why is PCIe so special that we *must* use a generic ID + quirks
rather than a specific ID?
Post by Jon Masters
Post by Ard Biesheuvel
If we do need a quirks handling mechanism, I still don't see how the
x86 situation extrapolates to ARM. ACPI offers plenty of ways for a
SoC vendor to identify the make and particular revision, and quirks
could be keyed off of that.
Fair enough. Is there any traction for an interim solution for these
initial platforms do you think? If we wait to add a new table it's
probably going to be the end of the year before we get this done.
The 'interim solution' is to come to terms with the fact that these
initial platforms are not SBSA compliant, contain a PCIe root complex
that is not PNP0408 but can be identified by its own HID, and we make
the software work with that. This means our message from the beginning
is that, yes, you can have non-compliant hardware and the burden is on
you to get it supported upstream, and no, you don't get to hang out
with the cool SBSA kids if you decide to go that
Ard Biesheuvel
2016-05-20 09:20:02 UTC
Permalink
Post by Jon Masters
Hi Ard
-----Original Message-----
[...]
Post by Jon Masters
Is the PCIe root complex so special that you cannot simply describe an
implementation that is not PNP0408 compatible as something else, under
its own unique HID? If everybody is onboard with using ACPI, how is
this any different from describing other parts of the platform
topology? Even if the SBSA mandates generic PCI, they already deviated
from that when they built the hardware, so pretending that it is a
PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow each
vendor to come up with his own code and it would be much more effort
for the PCI maintainer to rework the PCI framework to accommodate X86
and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this. Through
standardization we can more easily maintain the code and scale it to
multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
a bit more explanation...
OK, so that boils down to recommending to vendors to represent known
non-compliant hardware as compliant, just so that we don't have to
change the code to support additional flavors of ECAM ? It's fine to
be pragmatic, but that sucks.

We keep confusing the x86 case with the ARM case here: for x86, they
needed to deal with broken hardware *after* the fact, and all they
could do is find /some/ distinguishing feature in order to guess which
exact hardware they might be running on. For arm64, it is the opposite
case. We are currently in a position where we can demand vendors to
comply with the standards they endorsed themselves, and (ab)using ACPI
+ DMI as a de facto platform description rather than plain ACPI makes
me think the DT crowd were actually right from the beginning. It
*directly* violates the standardization principle, since it requires a
priori knowledge inside the OS that a certain 'generic' device must be
driven in a special way.

So can anyone comment on the feasibility of adding support for devices
with vendor specific HIDs (and no generic CIDs) to the current ACPI
ECAM driver in Linux?
Lorenzo Pieralisi
2016-05-23 11:00:02 UTC
Permalink
Post by Ard Biesheuvel
Post by Jon Masters
Hi Ard
-----Original Message-----
[...]
Post by Jon Masters
Is the PCIe root complex so special that you cannot simply describe an
implementation that is not PNP0408 compatible as something else, under
its own unique HID? If everybody is onboard with using ACPI, how is
this any different from describing other parts of the platform
topology? Even if the SBSA mandates generic PCI, they already deviated
from that when they built the hardware, so pretending that it is a
PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow each
vendor to come up with his own code and it would be much more effort
for the PCI maintainer to rework the PCI framework to accommodate X86
and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this. Through
standardization we can more easily maintain the code and scale it to
multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
a bit more explanation...
OK, so that boils down to recommending to vendors to represent known
non-compliant hardware as compliant, just so that we don't have to
change the code to support additional flavors of ECAM ? It's fine to
be pragmatic, but that sucks.
We keep confusing the x86 case with the ARM case here: for x86, they
needed to deal with broken hardware *after* the fact, and all they
could do is find /some/ distinguishing feature in order to guess which
exact hardware they might be running on. For arm64, it is the opposite
case. We are currently in a position where we can demand vendors to
comply with the standards they endorsed themselves, and (ab)using ACPI
+ DMI as a de facto platform description rather than plain ACPI makes
me think the DT crowd were actually right from the beginning. It
*directly* violates the standardization principle, since it requires a
priori knowledge inside the OS that a certain 'generic' device must be
driven in a special way.
So can anyone comment on the feasibility of adding support for devices
with vendor specific HIDs (and no generic CIDs) to the current ACPI
ECAM driver in Linux?
Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
most of the arch specific code is handled in the respective arch
directories (X86 and IA64, even though IA64 does not rely on ECAM/MCFG for
PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected through
ACPI scan handlers and the respective arch code (ie pci_acpi_scan_root)
sets-up resources AND config space on an arch specific basis.

X86 deals with that with code in arch/x86 that sets-up the pci_raw_ops
on a platform specific basis (and it is not nice, but it works because
as you all know the number of platforms in X86 world is contained).

Will this happen for ARM64 in arch/arm64 based on vendor specific
HIDs ?

No.

So given the current state of play (we were requested to move the
arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
with arch/arm64 code requiring code in /drivers to set-up pci_ops
in a platform specific way, it is horrible, if feasible at all.

The only way this can be implemented is by pretending that the
ACPI/PCI arch/arm64 implementation is generic code (that's what this
series does), move it to /drivers (where it is in this series), and
implement _DSD vendor specific bindings (per HID) to set-up the pci
operations; whether this solution should go upstream, given that it
is just a short-term solution for early platforms bugs, it is another
story and my personal answer is no.

Lorenzo
Gabriele Paoloni
2016-05-23 15:20:01 UTC
Permalink
Hi Lorenzo
-----Original Message-----
Sent: 23 May 2016 11:57
To: Ard Biesheuvel
Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
On 20 May 2016 at 10:40, Gabriele Paoloni
Post by Jon Masters
Hi Ard
-----Original Message-----
[...]
Post by Jon Masters
Is the PCIe root complex so special that you cannot simply
describe an
Post by Jon Masters
implementation that is not PNP0408 compatible as something else,
under
Post by Jon Masters
its own unique HID? If everybody is onboard with using ACPI, how
is
Post by Jon Masters
this any different from describing other parts of the platform
topology? Even if the SBSA mandates generic PCI, they already
deviated
Post by Jon Masters
from that when they built the hardware, so pretending that it is a
PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow
each
Post by Jon Masters
vendor to come up with his own code and it would be much more
effort
Post by Jon Masters
for the PCI maintainer to rework the PCI framework to accommodate
X86
Post by Jon Masters
and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this.
Through
Post by Jon Masters
standardization we can more easily maintain the code and scale it
to
Post by Jon Masters
multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
a bit more explanation...
OK, so that boils down to recommending to vendors to represent known
non-compliant hardware as compliant, just so that we don't have to
change the code to support additional flavors of ECAM ? It's fine to
be pragmatic, but that sucks.
We keep confusing the x86 case with the ARM case here: for x86, they
needed to deal with broken hardware *after* the fact, and all they
could do is find /some/ distinguishing feature in order to guess
which
exact hardware they might be running on. For arm64, it is the
opposite
case. We are currently in a position where we can demand vendors to
comply with the standards they endorsed themselves, and (ab)using
ACPI
+ DMI as a de facto platform description rather than plain ACPI makes
me think the DT crowd were actually right from the beginning. It
*directly* violates the standardization principle, since it requires
a
priori knowledge inside the OS that a certain 'generic' device must
be
driven in a special way.
So can anyone comment on the feasibility of adding support for
devices
with vendor specific HIDs (and no generic CIDs) to the current ACPI
ECAM driver in Linux?
Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
most of the arch specific code is handled in the respective arch
directories (X86 and IA64, even though IA64 does not rely on ECAM/MCFG for
PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected through
ACPI scan handlers and the respective arch code (ie pci_acpi_scan_root)
sets-up resources AND config space on an arch specific basis.
X86 deals with that with code in arch/x86 that sets-up the pci_raw_ops
on a platform specific basis (and it is not nice, but it works because
as you all know the number of platforms in X86 world is contained).
Will this happen for ARM64 in arch/arm64 based on vendor specific
HIDs ?
No.
So given the current state of play (we were requested to move the
arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
with arch/arm64 code requiring code in /drivers to set-up pci_ops
in a platform specific way, it is horrible, if feasible at all.
The only way this can be implemented is by pretending that the
ACPI/PCI arch/arm64 implementation is generic code (that's what this
series does), move it to /drivers (where it is in this series), and
implement _DSD vendor specific bindings (per HID) to set-up the pci
operations; whether this solution should go upstream, given that it
is just a short-term solution for early platforms bugs, it is another
story and my personal answer is no.
I think it shouldn't be too bad to move quirk handling mechanism to
arch/arm64. Effectively we would not move platform specific code into
arch/arm64 but just the mechanism checking if there is any quirk that
is defined.

i.e.:

extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];

static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root *root)
{
int bus_num = root->secondary.start;
int domain = root->segment;
struct pci_cfg_fixup *f;

/*
* Match against platform specific quirks and return corresponding
* CAM ops.
*
* First match against PCI topology <domain:bus> then use DMI or
* custom match handler.
*/
for (f = __start_acpi_mcfg_fixups; f < __end_acpi_mcfg_fixups; f++) {
if ((f->domain == domain || f->domain == PCI_MCFG_DOMAIN_ANY) &&
(f->bus_num == bus_num || f->bus_num == PCI_MCFG_BUS_ANY) &&
(f->system ? dmi_check_system(f->system) : 1) &&
(f->match ? f->match(f, root) : 1))
return f->ops;
}
/* No quirks, use ECAM */
return &pci_generic_ecam_ops;
}

Such quirks will be defined anyway in drivers/pci/host/ in the vendor
specific quirk implementations.

e.g. in HiSilicon case we would have

DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops,
PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);

in "drivers/pci/host/pcie-hisi-acpi.c "

Thanks

Gab
Lorenzo
Bjorn Helgaas
2016-05-23 23:40:02 UTC
Permalink
Post by Gabriele Paoloni
Hi Lorenzo
-----Original Message-----
Sent: 23 May 2016 11:57
To: Ard Biesheuvel
Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
On 20 May 2016 at 10:40, Gabriele Paoloni
Post by Jon Masters
Hi Ard
-----Original Message-----
[...]
Post by Jon Masters
Is the PCIe root complex so special that you cannot simply
describe an
Post by Jon Masters
implementation that is not PNP0408 compatible as something else,
under
Post by Jon Masters
its own unique HID? If everybody is onboard with using ACPI, how
is
Post by Jon Masters
this any different from describing other parts of the platform
topology? Even if the SBSA mandates generic PCI, they already
deviated
Post by Jon Masters
from that when they built the hardware, so pretending that it is a
PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow
each
Post by Jon Masters
vendor to come up with his own code and it would be much more
effort
Post by Jon Masters
for the PCI maintainer to rework the PCI framework to accommodate
X86
Post by Jon Masters
and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this.
Through
Post by Jon Masters
standardization we can more easily maintain the code and scale it
to
Post by Jon Masters
multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
a bit more explanation...
OK, so that boils down to recommending to vendors to represent known
non-compliant hardware as compliant, just so that we don't have to
change the code to support additional flavors of ECAM ? It's fine to
be pragmatic, but that sucks.
We keep confusing the x86 case with the ARM case here: for x86, they
needed to deal with broken hardware *after* the fact, and all they
could do is find /some/ distinguishing feature in order to guess
which
exact hardware they might be running on. For arm64, it is the
opposite
case. We are currently in a position where we can demand vendors to
comply with the standards they endorsed themselves, and (ab)using
ACPI
+ DMI as a de facto platform description rather than plain ACPI makes
me think the DT crowd were actually right from the beginning. It
*directly* violates the standardization principle, since it requires
a
priori knowledge inside the OS that a certain 'generic' device must
be
driven in a special way.
So can anyone comment on the feasibility of adding support for
devices
with vendor specific HIDs (and no generic CIDs) to the current ACPI
ECAM driver in Linux?
I don't think of ECAM support itself as a "driver". It's just a
service available to drivers, similar to OF resource parsing.

Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
supports extended config space. It doesn't specify how we access that
config space, so I think hardware with non-standard ECAM should still
have PNP0A03 and PNP0A08 in _CID or _HID.

"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
r3.2, sec 4.1) means:

(a) a memory-mapped model for config space access, and
(b) a specific mapping of address bits to bus/device/function/
register

MCFG and _CBA assume both (a) and (b), so I think a device with
non-standard ECAM mappings should not be described in MCFG or _CBA.

If a bridge has ECAM with non-standard mappings, I think either a
vendor-specific _HID or a device-specific method, e.g., _DSM, could
communicate that.

Jon, I agree that we should avoid describing non-standardized hardware
in Linux-specific ways. Is there a mechanism in use already? How
does Windows handle this? DMI is a poor long-term solution because it
requires ongoing maintenance for new platforms, but I think it's OK
for getting started with platforms already shipping.

A _DSM has the advantage that once it is defined and supported, OEMs
can ship new platforms without requiring a new quirk or a new _HID to
be added to a driver.

There would still be the problem of config access before the namespace
is available, i.e., the MCFG use case. I don't know how important
that is. Defining an MCFG extension seems like the most obvious
solution.

If we only expect a few non-standard devices, maybe it's enough to
have DMI quirks to statically set up ECAM and just live with the
inconvenience of requiring a kernel change for every new non-standard
device.
Post by Gabriele Paoloni
Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
most of the arch specific code is handled in the respective arch
directories (X86 and IA64, even though IA64 does not rely on ECAM/MCFG for
PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected through
ACPI scan handlers and the respective arch code (ie pci_acpi_scan_root)
sets-up resources AND config space on an arch specific basis.
X86 deals with that with code in arch/x86 that sets-up the pci_raw_ops
on a platform specific basis (and it is not nice, but it works because
as you all know the number of platforms in X86 world is contained).
Will this happen for ARM64 in arch/arm64 based on vendor specific
HIDs ?
No.
So given the current state of play (we were requested to move the
arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
with arch/arm64 code requiring code in /drivers to set-up pci_ops
in a platform specific way, it is horrible, if feasible at all.
The only way this can be implemented is by pretending that the
ACPI/PCI arch/arm64 implementation is generic code (that's what this
series does), move it to /drivers (where it is in this series), and
implement _DSD vendor specific bindings (per HID) to set-up the pci
operations; whether this solution should go upstream, given that it
is just a short-term solution for early platforms bugs, it is another
story and my personal answer is no.
It seems like there should be a way to look for a _DSM before we call
acpi_pci_root_get_mcfg_addr() to look for _CBA.

Currently we call acpi_pci_root_get_mcfg_addr() (to read _CBA) from
the generic acpi_pci_root_add(), but the result (root->mcfg_addr) is
only used in x86-specific code. I think it would be nicer if the
lookup and the use were together. Then it would be easier to override
it because the mapping assumptions would all be in one place.
Post by Gabriele Paoloni
I think it shouldn't be too bad to move quirk handling mechanism to
arch/arm64. Effectively we would not move platform specific code into
arch/arm64 but just the mechanism checking if there is any quirk that
is defined.
extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root *root)
{
int bus_num = root->secondary.start;
int domain = root->segment;
struct pci_cfg_fixup *f;
/*
* Match against platform specific quirks and return corresponding
* CAM ops.
*
* First match against PCI topology <domain:bus> then use DMI or
* custom match handler.
*/
for (f = __start_acpi_mcfg_fixups; f < __end_acpi_mcfg_fixups; f++) {
if ((f->domain == domain || f->domain == PCI_MCFG_DOMAIN_ANY) &&
(f->bus_num == bus_num || f->bus_num == PCI_MCFG_BUS_ANY) &&
(f->system ? dmi_check_system(f->system) : 1) &&
(f->match ? f->match(f, root) : 1))
return f->ops;
}
/* No quirks, use ECAM */
return &pci_generic_ecam_ops;
}
Such quirks will be defined anyway in drivers/pci/host/ in the vendor
specific quirk implementations.
e.g. in HiSilicon case we would have
DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops,
PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
in "drivers/pci/host/pcie-hisi-acpi.c "
Thanks
Gab
Sorry Gab, I guess I was really responding to earlier messages :)

I don't really have much to say here, except that it doesn't seem
right to have an MCFG that describes a non-standard ECAM mapping.
I suppose there's already firmware in the field that does that,
though?

Bjorn
Jon Masters
2016-05-24 01:20:01 UTC
Permalink
Bjorn,

Out walking so sorry about top posting. Quick reply though:

1. I checked with the Windows team. They usually avoid quirks entirely but when it has happened, it has been done via the MCFG/FADT not DSDT.

2. They would be ok if we were to key off the OEM name and revision for the IP in the MCFG table.

3. I have already verified existing shipping ARMv8 systems provide enough unique data in that entry, and have asked that vendors guarantee to rev it in future IP (which I will verify on models pre tapeout and certainly in early firmware builds). One vendor has a platform that isn't public yet that uses a non-public name in the MCFG but I spoke with them on Friday and they will shortly update their firmware so that a quirk could be posted.

4. I have requested (and Linaro are investigating) that Linaro (with ARM's assistance) begin to drive a separate thread around upstreaming (independent of this core effort) quirks that use the OEM fields in the MCFG as a more scalable approach than one per platform via DMI.

5. I will drive a clarification to the SBBR that does not encourage or endorse quirks but does merely reinforce that data must be unique in such tables. I am driving a separate series of conversations with vendors to ensure that this is the case on all future platforms - though just generally, there is no more high end top shelf "Xeon class" silicon needing common quirks in the pipeline.

More later.

Jon.
--
Computer Architect | Sent from my 64-bit #ARM Powered phone
Post by Bjorn Helgaas
Post by Gabriele Paoloni
Hi Lorenzo
-----Original Message-----
Sent: 23 May 2016 11:57
To: Ard Biesheuvel
Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
On 20 May 2016 at 10:40, Gabriele Paoloni
Post by Jon Masters
Hi Ard
-----Original Message-----
[...]
Post by Jon Masters
Is the PCIe root complex so special that you cannot simply
describe an
Post by Jon Masters
implementation that is not PNP0408 compatible as something else,
under
Post by Jon Masters
its own unique HID? If everybody is onboard with using ACPI, how
is
Post by Jon Masters
this any different from describing other parts of the platform
topology? Even if the SBSA mandates generic PCI, they already
deviated
Post by Jon Masters
from that when they built the hardware, so pretending that it is a
PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow
each
Post by Jon Masters
vendor to come up with his own code and it would be much more
effort
Post by Jon Masters
for the PCI maintainer to rework the PCI framework to accommodate
X86
Post by Jon Masters
and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this.
Through
Post by Jon Masters
standardization we can more easily maintain the code and scale it
to
Post by Jon Masters
multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
a bit more explanation...
OK, so that boils down to recommending to vendors to represent known
non-compliant hardware as compliant, just so that we don't have to
change the code to support additional flavors of ECAM ? It's fine to
be pragmatic, but that sucks.
We keep confusing the x86 case with the ARM case here: for x86, they
needed to deal with broken hardware *after* the fact, and all they
could do is find /some/ distinguishing feature in order to guess
which
exact hardware they might be running on. For arm64, it is the
opposite
case. We are currently in a position where we can demand vendors to
comply with the standards they endorsed themselves, and (ab)using
ACPI
+ DMI as a de facto platform description rather than plain ACPI makes
me think the DT crowd were actually right from the beginning. It
*directly* violates the standardization principle, since it requires
a
priori knowledge inside the OS that a certain 'generic' device must
be
driven in a special way.
So can anyone comment on the feasibility of adding support for
devices
with vendor specific HIDs (and no generic CIDs) to the current ACPI
ECAM driver in Linux?
I don't think of ECAM support itself as a "driver". It's just a
service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
supports extended config space. It doesn't specify how we access that
config space, so I think hardware with non-standard ECAM should still
have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
(a) a memory-mapped model for config space access, and
(b) a specific mapping of address bits to bus/device/function/
register
MCFG and _CBA assume both (a) and (b), so I think a device with
non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a
vendor-specific _HID or a device-specific method, e.g., _DSM, could
communicate that.
Jon, I agree that we should avoid describing non-standardized hardware
in Linux-specific ways. Is there a mechanism in use already? How
does Windows handle this? DMI is a poor long-term solution because it
requires ongoing maintenance for new platforms, but I think it's OK
for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs
can ship new platforms without requiring a new quirk or a new _HID to
be added to a driver.
There would still be the problem of config access before the namespace
is available, i.e., the MCFG use case. I don't know how important
that is. Defining an MCFG extension seems like the most obvious
solution.
If we only expect a few non-standard devices, maybe it's enough to
have DMI quirks to statically set up ECAM and just live with the
inconvenience of requiring a kernel change for every new non-standard
device.
Post by Gabriele Paoloni
Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
most of the arch specific code is handled in the respective arch
directories (X86 and IA64, even though IA64 does not rely on ECAM/MCFG for
PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected through
ACPI scan handlers and the respective arch code (ie pci_acpi_scan_root)
sets-up resources AND config space on an arch specific basis.
X86 deals with that with code in arch/x86 that sets-up the pci_raw_ops
on a platform specific basis (and it is not nice, but it works because
as you all know the number of platforms in X86 world is contained).
Will this happen for ARM64 in arch/arm64 based on vendor specific
HIDs ?
No.
So given the current state of play (we were requested to move the
arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
with arch/arm64 code requiring code in /drivers to set-up pci_ops
in a platform specific way, it is horrible, if feasible at all.
The only way this can be implemented is by pretending that the
ACPI/PCI arch/arm64 implementation is generic code (that's what this
series does), move it to /drivers (where it is in this series), and
implement _DSD vendor specific bindings (per HID) to set-up the pci
operations; whether this solution should go upstream, given that it
is just a short-term solution for early platforms bugs, it is another
story and my personal answer is no.
It seems like there should be a way to look for a _DSM before we call
acpi_pci_root_get_mcfg_addr() to look for _CBA.
Currently we call acpi_pci_root_get_mcfg_addr() (to read _CBA) from
the generic acpi_pci_root_add(), but the result (root->mcfg_addr) is
only used in x86-specific code. I think it would be nicer if the
lookup and the use were together. Then it would be easier to override
it because the mapping assumptions would all be in one place.
Post by Gabriele Paoloni
I think it shouldn't be too bad to move quirk handling mechanism to
arch/arm64. Effectively we would not move platform specific code into
arch/arm64 but just the mechanism checking if there is any quirk that
is defined.
extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root *root)
{
int bus_num = root->secondary.start;
int domain = root->segment;
struct pci_cfg_fixup *f;
/*
* Match against platform specific quirks and return corresponding
* CAM ops.
*
* First match against PCI topology <domain:bus> then use DMI or
* custom match handler.
*/
for (f = __start_acpi_mcfg_fixups; f < __end_acpi_mcfg_fixups; f++) {
if ((f->domain == domain || f->domain == PCI_MCFG_DOMAIN_ANY) &&
(f->bus_num == bus_num || f->bus_num == PCI_MCFG_BUS_ANY) &&
(f->system ? dmi_check_system(f->system) : 1) &&
(f->match ? f->match(f, root) : 1))
return f->ops;
}
/* No quirks, use ECAM */
return &pci_generic_ecam_ops;
}
Such quirks will be defined anyway in drivers/pci/host/ in the vendor
specific quirk implementations.
e.g. in HiSilicon case we would have
DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops,
PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
in "drivers/pci/host/pcie-hisi-acpi.c "
Thanks
Gab
Sorry Gab, I guess I was really responding to earlier messages :)
I don't really have much to say here, except that it doesn't seem
right to have an MCFG that describes a non-standard ECAM mapping.
I suppose there's already firmware in the field that does that,
though?
Bjorn
Jon Masters
2016-05-24 01:50:02 UTC
Permalink
Additional: I would like to thank Ard for suggesting this approach. It turns out (apparently) that Mark Salter's initial X-Gene quirks internal to RH did it this way as well. You great minds think alike. If this works for folks then I hope it leads to upstream kernel support in F25 (we have a bunch of Moonshot hardware we would like to deeply in Fedora but can't without the PCIe network...). I rant only because I care :)

Jon.
--
Computer Architect | Sent from my 64-bit #ARM Powered phone
Post by Jon Masters
Bjorn,
1. I checked with the Windows team. They usually avoid quirks entirely but when it has happened, it has been done via the MCFG/FADT not DSDT.
2. They would be ok if we were to key off the OEM name and revision for the IP in the MCFG table.
3. I have already verified existing shipping ARMv8 systems provide enough unique data in that entry, and have asked that vendors guarantee to rev it in future IP (which I will verify on models pre tapeout and certainly in early firmware builds). One vendor has a platform that isn't public yet that uses a non-public name in the MCFG but I spoke with them on Friday and they will shortly update their firmware so that a quirk could be posted.
4. I have requested (and Linaro are investigating) that Linaro (with ARM's assistance) begin to drive a separate thread around upstreaming (independent of this core effort) quirks that use the OEM fields in the MCFG as a more scalable approach than one per platform via DMI.
5. I will drive a clarification to the SBBR that does not encourage or endorse quirks but does merely reinforce that data must be unique in such tables. I am driving a separate series of conversations with vendors to ensure that this is the case on all future platforms - though just generally, there is no more high end top shelf "Xeon class" silicon needing common quirks in the pipeline.
More later.
Jon.
--
Computer Architect | Sent from my 64-bit #ARM Powered phone
Post by Bjorn Helgaas
Post by Gabriele Paoloni
Hi Lorenzo
-----Original Message-----
Sent: 23 May 2016 11:57
To: Ard Biesheuvel
Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
On 20 May 2016 at 10:40, Gabriele Paoloni
Post by Jon Masters
Hi Ard
-----Original Message-----
[...]
Post by Jon Masters
Is the PCIe root complex so special that you cannot simply
describe an
Post by Jon Masters
implementation that is not PNP0408 compatible as something else,
under
Post by Jon Masters
its own unique HID? If everybody is onboard with using ACPI, how
is
Post by Jon Masters
this any different from describing other parts of the platform
topology? Even if the SBSA mandates generic PCI, they already
deviated
Post by Jon Masters
from that when they built the hardware, so pretending that it is a
PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow
each
Post by Jon Masters
vendor to come up with his own code and it would be much more
effort
Post by Jon Masters
for the PCI maintainer to rework the PCI framework to accommodate
X86
Post by Jon Masters
and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this.
Through
Post by Jon Masters
standardization we can more easily maintain the code and scale it
to
Post by Jon Masters
multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can give
a bit more explanation...
OK, so that boils down to recommending to vendors to represent known
non-compliant hardware as compliant, just so that we don't have to
change the code to support additional flavors of ECAM ? It's fine to
be pragmatic, but that sucks.
We keep confusing the x86 case with the ARM case here: for x86, they
needed to deal with broken hardware *after* the fact, and all they
could do is find /some/ distinguishing feature in order to guess
which
exact hardware they might be running on. For arm64, it is the
opposite
case. We are currently in a position where we can demand vendors to
comply with the standards they endorsed themselves, and (ab)using
ACPI
+ DMI as a de facto platform description rather than plain ACPI makes
me think the DT crowd were actually right from the beginning. It
*directly* violates the standardization principle, since it requires
a
priori knowledge inside the OS that a certain 'generic' device must
be
driven in a special way.
So can anyone comment on the feasibility of adding support for
devices
with vendor specific HIDs (and no generic CIDs) to the current ACPI
ECAM driver in Linux?
I don't think of ECAM support itself as a "driver". It's just a
service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
supports extended config space. It doesn't specify how we access that
config space, so I think hardware with non-standard ECAM should still
have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
(a) a memory-mapped model for config space access, and
(b) a specific mapping of address bits to bus/device/function/
register
MCFG and _CBA assume both (a) and (b), so I think a device with
non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a
vendor-specific _HID or a device-specific method, e.g., _DSM, could
communicate that.
Jon, I agree that we should avoid describing non-standardized hardware
in Linux-specific ways. Is there a mechanism in use already? How
does Windows handle this? DMI is a poor long-term solution because it
requires ongoing maintenance for new platforms, but I think it's OK
for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs
can ship new platforms without requiring a new quirk or a new _HID to
be added to a driver.
There would still be the problem of config access before the namespace
is available, i.e., the MCFG use case. I don't know how important
that is. Defining an MCFG extension seems like the most obvious
solution.
If we only expect a few non-standard devices, maybe it's enough to
have DMI quirks to statically set up ECAM and just live with the
inconvenience of requiring a kernel change for every new non-standard
device.
Post by Gabriele Paoloni
Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
most of the arch specific code is handled in the respective arch
directories (X86 and IA64, even though IA64 does not rely on ECAM/MCFG for
PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected through
ACPI scan handlers and the respective arch code (ie pci_acpi_scan_root)
sets-up resources AND config space on an arch specific basis.
X86 deals with that with code in arch/x86 that sets-up the pci_raw_ops
on a platform specific basis (and it is not nice, but it works because
as you all know the number of platforms in X86 world is contained).
Will this happen for ARM64 in arch/arm64 based on vendor specific
HIDs ?
No.
So given the current state of play (we were requested to move the
arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
with arch/arm64 code requiring code in /drivers to set-up pci_ops
in a platform specific way, it is horrible, if feasible at all.
The only way this can be implemented is by pretending that the
ACPI/PCI arch/arm64 implementation is generic code (that's what this
series does), move it to /drivers (where it is in this series), and
implement _DSD vendor specific bindings (per HID) to set-up the pci
operations; whether this solution should go upstream, given that it
is just a short-term solution for early platforms bugs, it is another
story and my personal answer is no.
It seems like there should be a way to look for a _DSM before we call
acpi_pci_root_get_mcfg_addr() to look for _CBA.
Currently we call acpi_pci_root_get_mcfg_addr() (to read _CBA) from
the generic acpi_pci_root_add(), but the result (root->mcfg_addr) is
only used in x86-specific code. I think it would be nicer if the
lookup and the use were together. Then it would be easier to override
it because the mapping assumptions would all be in one place.
Post by Gabriele Paoloni
I think it shouldn't be too bad to move quirk handling mechanism to
arch/arm64. Effectively we would not move platform specific code into
arch/arm64 but just the mechanism checking if there is any quirk that
is defined.
extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root *root)
{
int bus_num = root->secondary.start;
int domain = root->segment;
struct pci_cfg_fixup *f;
/*
* Match against platform specific quirks and return corresponding
* CAM ops.
*
* First match against PCI topology <domain:bus> then use DMI or
* custom match handler.
*/
for (f = __start_acpi_mcfg_fixups; f < __end_acpi_mcfg_fixups; f++) {
if ((f->domain == domain || f->domain == PCI_MCFG_DOMAIN_ANY) &&
(f->bus_num == bus_num || f->bus_num == PCI_MCFG_BUS_ANY) &&
(f->system ? dmi_check_system(f->system) : 1) &&
(f->match ? f->match(f, root) : 1))
return f->ops;
}
/* No quirks, use ECAM */
return &pci_generic_ecam_ops;
}
Such quirks will be defined anyway in drivers/pci/host/ in the vendor
specific quirk implementations.
e.g. in HiSilicon case we would have
DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops,
PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
in "drivers/pci/host/pcie-hisi-acpi.c "
Thanks
Gab
Sorry Gab, I guess I was really responding to earlier messages :)
I don't really have much to say here, except that it doesn't seem
right to have an MCFG that describes a non-standard ECAM mapping.
I suppose there's already firmware in the field that does that,
though?
Bjorn
Gabriele Paoloni
2016-05-24 14:40:02 UTC
Permalink
Hi Jon
-----Original Message-----
Sent: 24 May 2016 02:11
To: Bjorn Helgaas
Cc: Gabriele Paoloni; Lorenzo Pieralisi; Ard Biesheuvel; Tomasz
Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
Bjorn,
1. I checked with the Windows team. They usually avoid quirks entirely
but when it has happened, it has been done via the MCFG/FADT not DSDT.
2. They would be ok if we were to key off the OEM name and revision
for the IP in the MCFG table.
I see some problems with this approach:

1) We would need to modify the ACPI specs to accommodate quirks in the MCFG,
correct?

2) Just adding OEM info would not fit some other people (like us for Designware
based solutions). In our case for example the addresses defined in the MCFG
are not compatible with the ones used by the Designware IP, therefore we would
also need specific quirk data

I think that we can use an approach where we use MCFG entries for the ECAM
address spaces and motherboard reserved resources for those BUSes that are
outside the MCFG table and therefore are non ECAM.

I think it is a more generic approach that would suit anybody and there
is no need to redefine the ACPI specs for MCFG... ?

Thanks

Gab
3. I have already verified existing shipping ARMv8 systems provide
enough unique data in that entry, and have asked that vendors guarantee
to rev it in future IP (which I will verify on models pre tapeout and
certainly in early firmware builds). One vendor has a platform that
isn't public yet that uses a non-public name in the MCFG but I spoke
with them on Friday and they will shortly update their firmware so that
a quirk could be posted.
4. I have requested (and Linaro are investigating) that Linaro (with
ARM's assistance) begin to drive a separate thread around upstreaming
(independent of this core effort) quirks that use the OEM fields in the
MCFG as a more scalable approach than one per platform via DMI.
5. I will drive a clarification to the SBBR that does not encourage or
endorse quirks but does merely reinforce that data must be unique in
such tables. I am driving a separate series of conversations with
vendors to ensure that this is the case on all future platforms -
though just generally, there is no more high end top shelf "Xeon class"
silicon needing common quirks in the pipeline.
More later.
Jon.
--
Computer Architect | Sent from my 64-bit #ARM Powered phone
Post by Bjorn Helgaas
Post by Gabriele Paoloni
Hi Lorenzo
-----Original Message-----
Sent: 23 May 2016 11:57
To: Ard Biesheuvel
Cc: Gabriele Paoloni; Jon Masters; Tomasz Nowicki;
Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI
host
Post by Bjorn Helgaas
Post by Gabriele Paoloni
controller
On 20 May 2016 at 10:40, Gabriele Paoloni
Post by Jon Masters
Hi Ard
-----Original Message-----
[...]
Post by Jon Masters
Is the PCIe root complex so special that you cannot simply
describe an
Post by Jon Masters
implementation that is not PNP0408 compatible as something else,
under
Post by Jon Masters
its own unique HID? If everybody is onboard with using ACPI, how
is
Post by Jon Masters
this any different from describing other parts of the platform
topology? Even if the SBSA mandates generic PCI, they already
deviated
Post by Jon Masters
from that when they built the hardware, so pretending that it is
a
Post by Bjorn Helgaas
Post by Gabriele Paoloni
Post by Jon Masters
PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow
each
Post by Jon Masters
vendor to come up with his own code and it would be much more
effort
Post by Jon Masters
for the PCI maintainer to rework the PCI framework to accommodate
X86
Post by Jon Masters
and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this.
Through
Post by Jon Masters
standardization we can more easily maintain the code and scale it
to
Post by Jon Masters
multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can
give
Post by Bjorn Helgaas
Post by Gabriele Paoloni
Post by Jon Masters
a bit more explanation...
OK, so that boils down to recommending to vendors to represent
known
Post by Bjorn Helgaas
Post by Gabriele Paoloni
non-compliant hardware as compliant, just so that we don't have to
change the code to support additional flavors of ECAM ? It's fine
to
Post by Bjorn Helgaas
Post by Gabriele Paoloni
be pragmatic, but that sucks.
We keep confusing the x86 case with the ARM case here: for x86,
they
Post by Bjorn Helgaas
Post by Gabriele Paoloni
needed to deal with broken hardware *after* the fact, and all they
could do is find /some/ distinguishing feature in order to guess
which
exact hardware they might be running on. For arm64, it is the
opposite
case. We are currently in a position where we can demand vendors
to
Post by Bjorn Helgaas
Post by Gabriele Paoloni
comply with the standards they endorsed themselves, and (ab)using
ACPI
+ DMI as a de facto platform description rather than plain ACPI
makes
Post by Bjorn Helgaas
Post by Gabriele Paoloni
me think the DT crowd were actually right from the beginning. It
*directly* violates the standardization principle, since it
requires
Post by Bjorn Helgaas
Post by Gabriele Paoloni
a
priori knowledge inside the OS that a certain 'generic' device
must
Post by Bjorn Helgaas
Post by Gabriele Paoloni
be
driven in a special way.
So can anyone comment on the feasibility of adding support for
devices
with vendor specific HIDs (and no generic CIDs) to the current
ACPI
Post by Bjorn Helgaas
Post by Gabriele Paoloni
ECAM driver in Linux?
I don't think of ECAM support itself as a "driver". It's just a
service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
supports extended config space. It doesn't specify how we access
that
Post by Bjorn Helgaas
config space, so I think hardware with non-standard ECAM should still
have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
(a) a memory-mapped model for config space access, and
(b) a specific mapping of address bits to bus/device/function/
register
MCFG and _CBA assume both (a) and (b), so I think a device with
non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a
vendor-specific _HID or a device-specific method, e.g., _DSM, could
communicate that.
Jon, I agree that we should avoid describing non-standardized
hardware
Post by Bjorn Helgaas
in Linux-specific ways. Is there a mechanism in use already? How
does Windows handle this? DMI is a poor long-term solution because
it
Post by Bjorn Helgaas
requires ongoing maintenance for new platforms, but I think it's OK
for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs
can ship new platforms without requiring a new quirk or a new _HID to
be added to a driver.
There would still be the problem of config access before the
namespace
Post by Bjorn Helgaas
is available, i.e., the MCFG use case. I don't know how important
that is. Defining an MCFG extension seems like the most obvious
solution.
If we only expect a few non-standard devices, maybe it's enough to
have DMI quirks to statically set up ECAM and just live with the
inconvenience of requiring a kernel change for every new non-standard
device.
Post by Gabriele Paoloni
Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
most of the arch specific code is handled in the respective arch
directories (X86 and IA64, even though IA64 does not rely on
ECAM/MCFG
Post by Bjorn Helgaas
Post by Gabriele Paoloni
for
PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected through
ACPI scan handlers and the respective arch code (ie
pci_acpi_scan_root)
Post by Bjorn Helgaas
Post by Gabriele Paoloni
sets-up resources AND config space on an arch specific basis.
X86 deals with that with code in arch/x86 that sets-up the
pci_raw_ops
Post by Bjorn Helgaas
Post by Gabriele Paoloni
on a platform specific basis (and it is not nice, but it works
because
Post by Bjorn Helgaas
Post by Gabriele Paoloni
as you all know the number of platforms in X86 world is contained).
Will this happen for ARM64 in arch/arm64 based on vendor specific
HIDs ?
No.
So given the current state of play (we were requested to move the
arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
with arch/arm64 code requiring code in /drivers to set-up pci_ops
in a platform specific way, it is horrible, if feasible at all.
The only way this can be implemented is by pretending that the
ACPI/PCI arch/arm64 implementation is generic code (that's what
this
Post by Bjorn Helgaas
Post by Gabriele Paoloni
series does), move it to /drivers (where it is in this series), and
implement _DSD vendor specific bindings (per HID) to set-up the pci
operations; whether this solution should go upstream, given that it
is just a short-term solution for early platforms bugs, it is
another
Post by Bjorn Helgaas
Post by Gabriele Paoloni
story and my personal answer is no.
It seems like there should be a way to look for a _DSM before we call
acpi_pci_root_get_mcfg_addr() to look for _CBA.
Currently we call acpi_pci_root_get_mcfg_addr() (to read _CBA) from
the generic acpi_pci_root_add(), but the result (root->mcfg_addr) is
only used in x86-specific code. I think it would be nicer if the
lookup and the use were together. Then it would be easier to
override
Post by Bjorn Helgaas
it because the mapping assumptions would all be in one place.
Post by Gabriele Paoloni
I think it shouldn't be too bad to move quirk handling mechanism to
arch/arm64. Effectively we would not move platform specific code
into
Post by Bjorn Helgaas
Post by Gabriele Paoloni
arch/arm64 but just the mechanism checking if there is any quirk
that
Post by Bjorn Helgaas
Post by Gabriele Paoloni
is defined.
extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root
*root)
Post by Bjorn Helgaas
Post by Gabriele Paoloni
{
int bus_num = root->secondary.start;
int domain = root->segment;
struct pci_cfg_fixup *f;
/*
* Match against platform specific quirks and return
corresponding
Post by Bjorn Helgaas
Post by Gabriele Paoloni
* CAM ops.
*
* First match against PCI topology <domain:bus> then use DMI
or
Post by Bjorn Helgaas
Post by Gabriele Paoloni
* custom match handler.
*/
for (f = __start_acpi_mcfg_fixups; f <
__end_acpi_mcfg_fixups; f++) {
Post by Bjorn Helgaas
Post by Gabriele Paoloni
if ((f->domain == domain || f->domain ==
PCI_MCFG_DOMAIN_ANY) &&
Post by Bjorn Helgaas
Post by Gabriele Paoloni
(f->bus_num == bus_num || f->bus_num ==
PCI_MCFG_BUS_ANY) &&
Post by Bjorn Helgaas
Post by Gabriele Paoloni
(f->system ? dmi_check_system(f->system) : 1) &&
(f->match ? f->match(f, root) : 1))
return f->ops;
}
/* No quirks, use ECAM */
return &pci_generic_ecam_ops;
}
Such quirks will be defined anyway in drivers/pci/host/ in the
vendor
Post by Bjorn Helgaas
Post by Gabriele Paoloni
specific quirk implementations.
e.g. in HiSilicon case we would have
DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops,
PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
in "drivers/pci/host/pcie-hisi-acpi.c "
Thanks
Gab
Sorry Gab, I guess I was really responding to earlier messages :)
I don't really have much to say here, except that it doesn't seem
right to have an MCFG that describes a non-standard ECAM mapping.
I suppose there's already firmware in the field that does that,
though?
Bjorn
Gabriele Paoloni
2016-05-24 07:30:01 UTC
Permalink
Hi Bjorn
-----Original Message-----
Sent: 24 May 2016 00:39
To: Gabriele Paoloni
Cc: Lorenzo Pieralisi; Ard Biesheuvel; Jon Masters; Tomasz Nowicki;
Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
Post by Gabriele Paoloni
Hi Lorenzo
-----Original Message-----
Sent: 23 May 2016 11:57
To: Ard Biesheuvel
Cc: Gabriele Paoloni; Jon Masters; Tomasz Nowicki;
Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI
host
Post by Gabriele Paoloni
controller
On 20 May 2016 at 10:40, Gabriele Paoloni
Post by Jon Masters
Hi Ard
-----Original Message-----
[...]
Post by Jon Masters
Is the PCIe root complex so special that you cannot simply
describe an
Post by Jon Masters
implementation that is not PNP0408 compatible as something
else,
Post by Gabriele Paoloni
under
Post by Jon Masters
its own unique HID? If everybody is onboard with using ACPI,
how
Post by Gabriele Paoloni
is
Post by Jon Masters
this any different from describing other parts of the platform
topology? Even if the SBSA mandates generic PCI, they already
deviated
Post by Jon Masters
from that when they built the hardware, so pretending that it
is a
Post by Gabriele Paoloni
Post by Jon Masters
PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow
each
Post by Jon Masters
vendor to come up with his own code and it would be much more
effort
Post by Jon Masters
for the PCI maintainer to rework the PCI framework to
accommodate
Post by Gabriele Paoloni
X86
Post by Jon Masters
and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this.
Through
Post by Jon Masters
standardization we can more easily maintain the code and scale
it
Post by Gabriele Paoloni
to
Post by Jon Masters
multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can
give
Post by Gabriele Paoloni
Post by Jon Masters
a bit more explanation...
OK, so that boils down to recommending to vendors to represent
known
Post by Gabriele Paoloni
non-compliant hardware as compliant, just so that we don't have
to
Post by Gabriele Paoloni
change the code to support additional flavors of ECAM ? It's fine
to
Post by Gabriele Paoloni
be pragmatic, but that sucks.
We keep confusing the x86 case with the ARM case here: for x86,
they
Post by Gabriele Paoloni
needed to deal with broken hardware *after* the fact, and all
they
Post by Gabriele Paoloni
could do is find /some/ distinguishing feature in order to guess
which
exact hardware they might be running on. For arm64, it is the
opposite
case. We are currently in a position where we can demand vendors
to
Post by Gabriele Paoloni
comply with the standards they endorsed themselves, and (ab)using
ACPI
+ DMI as a de facto platform description rather than plain ACPI
makes
Post by Gabriele Paoloni
me think the DT crowd were actually right from the beginning. It
*directly* violates the standardization principle, since it
requires
Post by Gabriele Paoloni
a
priori knowledge inside the OS that a certain 'generic' device
must
Post by Gabriele Paoloni
be
driven in a special way.
So can anyone comment on the feasibility of adding support for
devices
with vendor specific HIDs (and no generic CIDs) to the current
ACPI
Post by Gabriele Paoloni
ECAM driver in Linux?
I don't think of ECAM support itself as a "driver". It's just a
service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
supports extended config space. It doesn't specify how we access that
config space, so I think hardware with non-standard ECAM should still
have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
(a) a memory-mapped model for config space access, and
(b) a specific mapping of address bits to bus/device/function/
register
MCFG and _CBA assume both (a) and (b), so I think a device with
non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a
vendor-specific _HID or a device-specific method, e.g., _DSM, could
communicate that.
Jon, I agree that we should avoid describing non-standardized hardware
in Linux-specific ways. Is there a mechanism in use already? How
does Windows handle this? DMI is a poor long-term solution because it
requires ongoing maintenance for new platforms, but I think it's OK
for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs
can ship new platforms without requiring a new quirk or a new _HID to
be added to a driver.
There would still be the problem of config access before the namespace
is available, i.e., the MCFG use case. I don't know how important
that is. Defining an MCFG extension seems like the most obvious
solution.
If we only expect a few non-standard devices, maybe it's enough to
have DMI quirks to statically set up ECAM and just live with the
inconvenience of requiring a kernel change for every new non-standard
device.
Post by Gabriele Paoloni
Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and
most of the arch specific code is handled in the respective arch
directories (X86 and IA64, even though IA64 does not rely on
ECAM/MCFG
Post by Gabriele Paoloni
for
PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected through
ACPI scan handlers and the respective arch code (ie
pci_acpi_scan_root)
Post by Gabriele Paoloni
sets-up resources AND config space on an arch specific basis.
X86 deals with that with code in arch/x86 that sets-up the
pci_raw_ops
Post by Gabriele Paoloni
on a platform specific basis (and it is not nice, but it works
because
Post by Gabriele Paoloni
as you all know the number of platforms in X86 world is contained).
Will this happen for ARM64 in arch/arm64 based on vendor specific
HIDs ?
No.
So given the current state of play (we were requested to move the
arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up
with arch/arm64 code requiring code in /drivers to set-up pci_ops
in a platform specific way, it is horrible, if feasible at all.
The only way this can be implemented is by pretending that the
ACPI/PCI arch/arm64 implementation is generic code (that's what
this
Post by Gabriele Paoloni
series does), move it to /drivers (where it is in this series), and
implement _DSD vendor specific bindings (per HID) to set-up the pci
operations; whether this solution should go upstream, given that it
is just a short-term solution for early platforms bugs, it is
another
Post by Gabriele Paoloni
story and my personal answer is no.
It seems like there should be a way to look for a _DSM before we call
acpi_pci_root_get_mcfg_addr() to look for _CBA.
Currently we call acpi_pci_root_get_mcfg_addr() (to read _CBA) from
the generic acpi_pci_root_add(), but the result (root->mcfg_addr) is
only used in x86-specific code. I think it would be nicer if the
lookup and the use were together. Then it would be easier to override
it because the mapping assumptions would all be in one place.
Post by Gabriele Paoloni
I think it shouldn't be too bad to move quirk handling mechanism to
arch/arm64. Effectively we would not move platform specific code into
arch/arm64 but just the mechanism checking if there is any quirk that
is defined.
extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[];
extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root
*root)
Post by Gabriele Paoloni
{
int bus_num = root->secondary.start;
int domain = root->segment;
struct pci_cfg_fixup *f;
/*
* Match against platform specific quirks and return
corresponding
Post by Gabriele Paoloni
* CAM ops.
*
* First match against PCI topology <domain:bus> then use DMI
or
Post by Gabriele Paoloni
* custom match handler.
*/
for (f = __start_acpi_mcfg_fixups; f <
__end_acpi_mcfg_fixups; f++) {
Post by Gabriele Paoloni
if ((f->domain == domain || f->domain ==
PCI_MCFG_DOMAIN_ANY) &&
Post by Gabriele Paoloni
(f->bus_num == bus_num || f->bus_num ==
PCI_MCFG_BUS_ANY) &&
Post by Gabriele Paoloni
(f->system ? dmi_check_system(f->system) : 1) &&
(f->match ? f->match(f, root) : 1))
return f->ops;
}
/* No quirks, use ECAM */
return &pci_generic_ecam_ops;
}
Such quirks will be defined anyway in drivers/pci/host/ in the vendor
specific quirk implementations.
e.g. in HiSilicon case we would have
DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops,
PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
in "drivers/pci/host/pcie-hisi-acpi.c "
Thanks
Gab
Sorry Gab, I guess I was really responding to earlier messages :)
I don't really have much to say here, except that it doesn't seem
right to have an MCFG that describes a non-standard ECAM mapping.
The ACPI table that this mechanism relies upon is the one discussed
in:
https://lkml.org/lkml/2016/3/9/91

As you can see MCFG describes ECAM mappings, but we have a motherboard
reserved resource outside the MCFG:
Device (RES0)
{
Name (_HID, "HISI0081") // HiSi PCIe RC config base address
Name (_CID, "PNP0C02") // Motherboard reserved resource
Name (_CRS, ResourceTemplate (){
Memory32Fixed (ReadWrite, 0xb0080000 , 0x10000)
})
}

This allows us to retrieve the address we need for accessing
the config space on the RC (that is non-ECAM).

I was thinking that such mechanism could fit different vendors
and allow them define their own quirks without spoiling the
official and standard MCFG; also from the thread discussion
you seemed quite ok with such solution...?
I suppose there's already firmware in the field that does that,
though?
We have "experimental" firmware that is based on the ACPI table
described above, however it is not widely distributed and, obviously,
it is not supported by Linux mainline (so we have room to rework
if we decide that another solution is more appropriate)

Thanks and Regards

Gab
Bjorn
Jon Masters
2016-05-24 14:40:03 UTC
Permalink
Hi Gabriele, all,
Post by Gabriele Paoloni
Post by Bjorn Helgaas
Sorry Gab, I guess I was really responding to earlier messages :)
I don't really have much to say here, except that it doesn't seem
right to have an MCFG that describes a non-standard ECAM mapping.
The ACPI table that this mechanism relies upon is the one discussed
https://lkml.org/lkml/2016/3/9/91
As you can see MCFG describes ECAM mappings, but we have a motherboard
Device (RES0)
{
Name (_HID, "HISI0081") // HiSi PCIe RC config base address
Name (_CID, "PNP0C02") // Motherboard reserved resource
Name (_CRS, ResourceTemplate (){
Memory32Fixed (ReadWrite, 0xb0080000 , 0x10000)
})
}
This allows us to retrieve the address we need for accessing
the config space on the RC (that is non-ECAM).
I was thinking that such mechanism could fit different vendors
and allow them define their own quirks without spoiling the
official and standard MCFG; also from the thread discussion
you seemed quite ok with such solution...?
This could have been useful 2-3 years ago (when myself and others first
pulled the fire alarm concerning the lack of upstreaming of the ACPI
enablement for PCIe - which should have been fully upstream before the
first platforms ever even shipped) but at this time we have shipping
platforms that don't have tables built in this way. While we can go back
around to vendors and try to get them to rebuild firmware, it would be
by far preferable to adopt a solution that works with what is already
being deployed in the field today. Such as OEM match in MCFG.

Jon.
--
Computer Architect | Sent from my Fedora powered laptop
Lorenzo Pieralisi
2016-05-24 17:30:01 UTC
Permalink
Hi Bjorn,

On Mon, May 23, 2016 at 06:39:18PM -0500, Bjorn Helgaas wrote:

[...]
Post by Bjorn Helgaas
I don't think of ECAM support itself as a "driver". It's just a
service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
supports extended config space. It doesn't specify how we access that
config space, so I think hardware with non-standard ECAM should still
have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
(a) a memory-mapped model for config space access, and
(b) a specific mapping of address bits to bus/device/function/
register
MCFG and _CBA assume both (a) and (b), so I think a device with
non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a
vendor-specific _HID or a device-specific method, e.g., _DSM, could
communicate that.
Jon, I agree that we should avoid describing non-standardized hardware
in Linux-specific ways. Is there a mechanism in use already? How
does Windows handle this? DMI is a poor long-term solution because it
requires ongoing maintenance for new platforms, but I think it's OK
for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs
can ship new platforms without requiring a new quirk or a new _HID to
be added to a driver.
There would still be the problem of config access before the namespace
is available, i.e., the MCFG use case. I don't know how important
that is. Defining an MCFG extension seems like the most obvious
solution.
Your summary above is a perfect representation of the situation.

We had an opportunity to sync-up on the current status of ACPI PCI
for ARM64 (and talked about a way forward for this series, which
includes quirks handling), let me summarize it here for everyone
involved so that we can agree on a way forward.

1) ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG
ECAM for config space is basically ready (Tomasz and JC addressed
Rafael's concerns in relation to ARM64 specific code, and managed
to find a way to allocate domain numbers in preparation for Arnd
pci_create_root_bus() clean-up, v8 to be posted shortly and should
be final). This provides support for de-facto ACPI/PCI ECAM base
standard for ARM64 (with a clean-split between generic code and ARM64
bits, where ARM64, like X86 and IA64, manages in arch code IO space and
PCI resources, to be further consolidated in the near future).
I do not think anyone can complain about the generality of what we
achieved, for systems that are PCI standard (yes, PCI STANDARD) this
would just be sufficient.
2) In a real world (1) is not enough. Some ARM64 platforms, not entirely
ECAM compliant, already shipped with the corresponding firmware that
we can't update. HW has ECAM quirks and to work around it in the kernel
we put forward many solutions to the problem, it is time we found a
solution (when, of course, (1) is completed and upstream).
Using the MCFG table OEMID matching floated around in this thread
would work fine for most of the platforms (and cross-OS) that have
shipped with HW ECAM quirks, so I think that's the starting point for
our solution and that's how we can sort this out, _today_.

The solution is a trivial look-up table:
MCFG OEMID <-> PCI config space ops

3) (2) does not just work on some platforms (and we can't predict the
future either - actually I can, it is three letters, ECAM), simply
because MCFG OEMID matching does not provide a way to attach further
data to the MCFG (eg if config space for, say, bus 0 domain 0, is not
ECAM compliant, the config region can't be handled and must not be
handled through a corresponding MCFG region.
That's the problem Gabriele is facing and wants to solve through
something like:

https://lkml.org/lkml/2016/3/9/91

in the respective ACPI tables-bindings. It may be an idea worth
pursuing, it does not solve (2) simply because that FW has shipped,
we can't patch it any longer.

Hence to finally support ACPI PCI on ARM64 I suggest we carry out the
following steps, in order:

- Let's complete/merge (1), that's fundamental to this whole thread
- On top of (1) we apply a quirking mechanism based on (2) that allows
us to boot mainline with boxes shipping today with no FW update required.
- We devise a way to handle quirks that is more generic than (2) so that
can we can accomodate further platforms that can't rely on (2) but
have more leeway in terms of FW updates.

I hope that's a reasonable plan, Tomasz's v8 series coming to kick it off.

Thank you,
Lorenzo
Jon Masters
2016-05-24 17:40:02 UTC
Permalink
A big +1 to the below :) :) :)
--
Computer Architect | Sent from my 64-bit #ARM Powered phone
Post by Lorenzo Pieralisi
Hi Bjorn,
[...]
Post by Bjorn Helgaas
I don't think of ECAM support itself as a "driver". It's just a
service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
supports extended config space. It doesn't specify how we access that
config space, so I think hardware with non-standard ECAM should still
have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
(a) a memory-mapped model for config space access, and
(b) a specific mapping of address bits to bus/device/function/
register
MCFG and _CBA assume both (a) and (b), so I think a device with
non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a
vendor-specific _HID or a device-specific method, e.g., _DSM, could
communicate that.
Jon, I agree that we should avoid describing non-standardized hardware
in Linux-specific ways. Is there a mechanism in use already? How
does Windows handle this? DMI is a poor long-term solution because it
requires ongoing maintenance for new platforms, but I think it's OK
for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs
can ship new platforms without requiring a new quirk or a new _HID to
be added to a driver.
There would still be the problem of config access before the namespace
is available, i.e., the MCFG use case. I don't know how important
that is. Defining an MCFG extension seems like the most obvious
solution.
Your summary above is a perfect representation of the situation.
We had an opportunity to sync-up on the current status of ACPI PCI
for ARM64 (and talked about a way forward for this series, which
includes quirks handling), let me summarize it here for everyone
involved so that we can agree on a way forward.
1) ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG
ECAM for config space is basically ready (Tomasz and JC addressed
Rafael's concerns in relation to ARM64 specific code, and managed
to find a way to allocate domain numbers in preparation for Arnd
pci_create_root_bus() clean-up, v8 to be posted shortly and should
be final). This provides support for de-facto ACPI/PCI ECAM base
standard for ARM64 (with a clean-split between generic code and ARM64
bits, where ARM64, like X86 and IA64, manages in arch code IO space and
PCI resources, to be further consolidated in the near future).
I do not think anyone can complain about the generality of what we
achieved, for systems that are PCI standard (yes, PCI STANDARD) this
would just be sufficient.
2) In a real world (1) is not enough. Some ARM64 platforms, not entirely
ECAM compliant, already shipped with the corresponding firmware that
we can't update. HW has ECAM quirks and to work around it in the kernel
we put forward many solutions to the problem, it is time we found a
solution (when, of course, (1) is completed and upstream).
Using the MCFG table OEMID matching floated around in this thread
would work fine for most of the platforms (and cross-OS) that have
shipped with HW ECAM quirks, so I think that's the starting point for
our solution and that's how we can sort this out, _today_.
MCFG OEMID <-> PCI config space ops
3) (2) does not just work on some platforms (and we can't predict the
future either - actually I can, it is three letters, ECAM), simply
because MCFG OEMID matching does not provide a way to attach further
data to the MCFG (eg if config space for, say, bus 0 domain 0, is not
ECAM compliant, the config region can't be handled and must not be
handled through a corresponding MCFG region.
That's the problem Gabriele is facing and wants to solve through
https://lkml.org/lkml/2016/3/9/91
in the respective ACPI tables-bindings. It may be an idea worth
pursuing, it does not solve (2) simply because that FW has shipped,
we can't patch it any longer.
Hence to finally support ACPI PCI on ARM64 I suggest we carry out the
- Let's complete/merge (1), that's fundamental to this whole thread
- On top of (1) we apply a quirking mechanism based on (2) that allows
us to boot mainline with boxes shipping today with no FW update required.
- We devise a way to handle quirks that is more generic than (2) so that
can we can accomodate further platforms that can't rely on (2) but
have more leeway in terms of FW updates.
I hope that's a reasonable plan, Tomasz's v8 series coming to kick it off.
Thank you,
Lorenzo
Bjorn Helgaas
2016-05-24 19:10:02 UTC
Permalink
Post by Lorenzo Pieralisi
Hi Bjorn,
[...]
Post by Bjorn Helgaas
I don't think of ECAM support itself as a "driver". It's just a
service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
supports extended config space. It doesn't specify how we access that
config space, so I think hardware with non-standard ECAM should still
have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
(a) a memory-mapped model for config space access, and
(b) a specific mapping of address bits to bus/device/function/
register
MCFG and _CBA assume both (a) and (b), so I think a device with
non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a
vendor-specific _HID or a device-specific method, e.g., _DSM, could
communicate that.
Jon, I agree that we should avoid describing non-standardized hardware
in Linux-specific ways. Is there a mechanism in use already? How
does Windows handle this? DMI is a poor long-term solution because it
requires ongoing maintenance for new platforms, but I think it's OK
for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs
can ship new platforms without requiring a new quirk or a new _HID to
be added to a driver.
There would still be the problem of config access before the namespace
is available, i.e., the MCFG use case. I don't know how important
that is. Defining an MCFG extension seems like the most obvious
solution.
Your summary above is a perfect representation of the situation.
We had an opportunity to sync-up on the current status of ACPI PCI
for ARM64 (and talked about a way forward for this series, which
includes quirks handling), let me summarize it here for everyone
involved so that we can agree on a way forward.
1) ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG
ECAM for config space is basically ready (Tomasz and JC addressed
Rafael's concerns in relation to ARM64 specific code, and managed
to find a way to allocate domain numbers in preparation for Arnd
pci_create_root_bus() clean-up, v8 to be posted shortly and should
be final). This provides support for de-facto ACPI/PCI ECAM base
standard for ARM64 (with a clean-split between generic code and ARM64
bits, where ARM64, like X86 and IA64, manages in arch code IO space and
PCI resources, to be further consolidated in the near future).
I do not think anyone can complain about the generality of what we
achieved, for systems that are PCI standard (yes, PCI STANDARD) this
would just be sufficient.
Sounds good to me.
Post by Lorenzo Pieralisi
2) In a real world (1) is not enough. Some ARM64 platforms, not entirely
ECAM compliant, already shipped with the corresponding firmware that
we can't update. HW has ECAM quirks and to work around it in the kernel
we put forward many solutions to the problem, it is time we found a
solution (when, of course, (1) is completed and upstream).
Using the MCFG table OEMID matching floated around in this thread
would work fine for most of the platforms (and cross-OS) that have
shipped with HW ECAM quirks, so I think that's the starting point for
our solution and that's how we can sort this out, _today_.
MCFG OEMID <-> PCI config space ops
Sounds reasonable to me.
Post by Lorenzo Pieralisi
3) (2) does not just work on some platforms (and we can't predict the
future either - actually I can, it is three letters, ECAM), simply
because MCFG OEMID matching does not provide a way to attach further
data to the MCFG (eg if config space for, say, bus 0 domain 0, is not
ECAM compliant, the config region can't be handled and must not be
handled through a corresponding MCFG region.
Couldn't this be handled by custom pci_ops that do something special
for bus 0 domain 0, and default to some different pci_ops for the
rest?
Post by Lorenzo Pieralisi
That's the problem Gabriele is facing and wants to solve through
https://lkml.org/lkml/2016/3/9/91
in the respective ACPI tables-bindings. It may be an idea worth
pursuing, it does not solve (2) simply because that FW has shipped,
we can't patch it any longer.
(2) is for quirks to deal with MCFG. Gabriele's post is a proposal
for ACPI namespace. We can't use anything in the namespace to
implement MCFG quirks because MCFG is needed before the namespace is
available.

I think Gabriele's post is a good proposal for the namespace, but I
would propose the following modifications:

- Add PNP0A08 to the PCI1 _CID since this is a PCIe host bridge
- Add a PCI1 _DSM describing the ECAM space
- Remove the HISI0081 _HID

The result would be:

Device (PCI1) {
Name(_HID, "HISI0080")
Name(_CID, "PNP0A03,PNP0A08")
Method(_CRS) { ... }
Method(_DSM) { <describe ECAM base and mapping function> }
}
Device (RES0) {
Name(_HID, "PNP0C02")
Name(_CRS) { <describe ECAM base and size> }
}

RES0 could also be contained within PCI1, as Gabriele suggested. I
don't really care whether it's contained or not, and making it
contained might make it easier for firmware, because addition/removal
of PCI1 and RES0 should always happen together.

I think the _DSM is important because it is really ugly if the
HISI0080 driver has to look for a separate HISI0081 device to learn
about the ECAM space. There are several PCI drivers that do something
similar, using for_each_pci_dev(), and I cringe every time I see them,
because this totally screws up the driver model. A driver should
claim a device via a .probe() method called by the core, and it
shouldn't look at devices it hasn't claimed. This is required to make
hotplug work correctly.
Post by Lorenzo Pieralisi
Hence to finally support ACPI PCI on ARM64 I suggest we carry out the
- Let's complete/merge (1), that's fundamental to this whole thread
- On top of (1) we apply a quirking mechanism based on (2) that allows
us to boot mainline with boxes shipping today with no FW update required.
- We devise a way to handle quirks that is more generic than (2) so that
can we can accomodate further platforms that can't rely on (2) but
have more leeway in terms of FW updates.
I hope that's a reasonable plan, Tomasz's v8 series coming to kick it off.
Sounds very good to me; I'm looking forward to v8.

Bjorn
Gabriele Paoloni
2016-05-26 10:00:01 UTC
Permalink
Hi Bjorn many thanks for your suggestions
-----Original Message-----
Sent: 24 May 2016 20:01
To: Lorenzo Pieralisi
Cc: Gabriele Paoloni; Ard Biesheuvel; Jon Masters; Tomasz Nowicki;
Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
Post by Lorenzo Pieralisi
Hi Bjorn,
[...]
Post by Bjorn Helgaas
I don't think of ECAM support itself as a "driver". It's just a
service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
supports extended config space. It doesn't specify how we access
that
Post by Lorenzo Pieralisi
Post by Bjorn Helgaas
config space, so I think hardware with non-standard ECAM should
still
Post by Lorenzo Pieralisi
Post by Bjorn Helgaas
have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
(a) a memory-mapped model for config space access, and
(b) a specific mapping of address bits to bus/device/function/
register
MCFG and _CBA assume both (a) and (b), so I think a device with
non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a
vendor-specific _HID or a device-specific method, e.g., _DSM, could
communicate that.
Jon, I agree that we should avoid describing non-standardized
hardware
Post by Lorenzo Pieralisi
Post by Bjorn Helgaas
in Linux-specific ways. Is there a mechanism in use already? How
does Windows handle this? DMI is a poor long-term solution because
it
Post by Lorenzo Pieralisi
Post by Bjorn Helgaas
requires ongoing maintenance for new platforms, but I think it's OK
for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported,
OEMs
Post by Lorenzo Pieralisi
Post by Bjorn Helgaas
can ship new platforms without requiring a new quirk or a new _HID
to
Post by Lorenzo Pieralisi
Post by Bjorn Helgaas
be added to a driver.
There would still be the problem of config access before the
namespace
Post by Lorenzo Pieralisi
Post by Bjorn Helgaas
is available, i.e., the MCFG use case. I don't know how important
that is. Defining an MCFG extension seems like the most obvious
solution.
Your summary above is a perfect representation of the situation.
We had an opportunity to sync-up on the current status of ACPI PCI
for ARM64 (and talked about a way forward for this series, which
includes quirks handling), let me summarize it here for everyone
involved so that we can agree on a way forward.
1) ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG
ECAM for config space is basically ready (Tomasz and JC addressed
Rafael's concerns in relation to ARM64 specific code, and managed
to find a way to allocate domain numbers in preparation for Arnd
pci_create_root_bus() clean-up, v8 to be posted shortly and should
be final). This provides support for de-facto ACPI/PCI ECAM base
standard for ARM64 (with a clean-split between generic code and
ARM64
Post by Lorenzo Pieralisi
bits, where ARM64, like X86 and IA64, manages in arch code IO
space and
Post by Lorenzo Pieralisi
PCI resources, to be further consolidated in the near future).
I do not think anyone can complain about the generality of what we
achieved, for systems that are PCI standard (yes, PCI STANDARD)
this
Post by Lorenzo Pieralisi
would just be sufficient.
Sounds good to me.
Post by Lorenzo Pieralisi
2) In a real world (1) is not enough. Some ARM64 platforms, not
entirely
Post by Lorenzo Pieralisi
ECAM compliant, already shipped with the corresponding firmware
that
Post by Lorenzo Pieralisi
we can't update. HW has ECAM quirks and to work around it in the
kernel
Post by Lorenzo Pieralisi
we put forward many solutions to the problem, it is time we found
a
Post by Lorenzo Pieralisi
solution (when, of course, (1) is completed and upstream).
Using the MCFG table OEMID matching floated around in this thread
would work fine for most of the platforms (and cross-OS) that have
shipped with HW ECAM quirks, so I think that's the starting point
for
Post by Lorenzo Pieralisi
our solution and that's how we can sort this out, _today_.
MCFG OEMID <-> PCI config space ops
Sounds reasonable to me.
Post by Lorenzo Pieralisi
3) (2) does not just work on some platforms (and we can't predict the
future either - actually I can, it is three letters, ECAM), simply
because MCFG OEMID matching does not provide a way to attach
further
Post by Lorenzo Pieralisi
data to the MCFG (eg if config space for, say, bus 0 domain 0, is
not
Post by Lorenzo Pieralisi
ECAM compliant, the config region can't be handled and must not be
handled through a corresponding MCFG region.
Couldn't this be handled by custom pci_ops that do something special
for bus 0 domain 0, and default to some different pci_ops for the
rest?
My idea was to remove bus 0 from domain 0 MCFG (and also the other
buses corresponding to the root complexes ports)

So the driver would use the ECAM access with addresses retrieved from
MCFG for any devices except the RCs.

For the RC we could retrieve special addresses from the "PNP0C02"
reserved resource...
Post by Lorenzo Pieralisi
That's the problem Gabriele is facing and wants to solve through
https://lkml.org/lkml/2016/3/9/91
in the respective ACPI tables-bindings. It may be an idea worth
pursuing, it does not solve (2) simply because that FW has
shipped,
Post by Lorenzo Pieralisi
we can't patch it any longer.
(2) is for quirks to deal with MCFG. Gabriele's post is a proposal
for ACPI namespace. We can't use anything in the namespace to
implement MCFG quirks because MCFG is needed before the namespace is
available.
Honestly I have to dig better into this (and I will once V8 is out),
However from my current understanding so far we have look-up where
MCFG OEMID is going to tell which specific pci-ops are going to be used.

Now from my quirk idea I need to retrieve the "special address" from the
ACPI namespace before cfg rd/wr take place...if this is not doable then
I need to find a different solution...than can be the one you proposed
below (many thanks for this).

I will look into details later on once v8 is out.

Thanks

Gab
I think Gabriele's post is a good proposal for the namespace, but I
- Add PNP0A08 to the PCI1 _CID since this is a PCIe host bridge
- Add a PCI1 _DSM describing the ECAM space
- Remove the HISI0081 _HID
Device (PCI1) {
Name(_HID, "HISI0080")
Name(_CID, "PNP0A03,PNP0A08")
Method(_CRS) { ... }
Method(_DSM) { <describe ECAM base and mapping function> }
}
Device (RES0) {
Name(_HID, "PNP0C02")
Name(_CRS) { <describe ECAM base and size> }
}
RES0 could also be contained within PCI1, as Gabriele suggested. I
don't really care whether it's contained or not, and making it
contained might make it easier for firmware, because addition/removal
of PCI1 and RES0 should always happen together.
I think the _DSM is important because it is really ugly if the
HISI0080 driver has to look for a separate HISI0081 device to learn
about the ECAM space. There are several PCI drivers that do something
similar, using for_each_pci_dev(), and I cringe every time I see them,
because this totally screws up the driver model. A driver should
claim a device via a .probe() method called by the core, and it
shouldn't look at devices it hasn't claimed. This is required to make
hotplug work correctly.
Post by Lorenzo Pieralisi
Hence to finally support ACPI PCI on ARM64 I suggest we carry out the
- Let's complete/merge (1), that's fundamental to this whole thread
- On top of (1) we apply a quirking mechanism based on (2) that
allows
Post by Lorenzo Pieralisi
us to boot mainline with boxes shipping today with no FW update
required.
Post by Lorenzo Pieralisi
- We devise a way to handle quirks that is more generic than (2) so
that
Post by Lorenzo Pieralisi
can we can accomodate further platforms that can't rely on (2) but
have more leeway in terms of FW updates.
I hope that's a reasonable plan, Tomasz's v8 series coming to kick it
off.
Sounds very good to me; I'm looking forward to v8.
Bjorn
Gabriele Paoloni
2016-05-25 06:40:01 UTC
Permalink
Hi Lorenzo
-----Original Message-----
Sent: 24 May 2016 18:24
To: Bjorn Helgaas
Cc: Gabriele Paoloni; Ard Biesheuvel; Jon Masters; Tomasz Nowicki;
Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
Hi Bjorn,
[...]
Post by Bjorn Helgaas
I don't think of ECAM support itself as a "driver". It's just a
service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe
host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that
supports extended config space. It doesn't specify how we access
that
Post by Bjorn Helgaas
config space, so I think hardware with non-standard ECAM should still
have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware
(a) a memory-mapped model for config space access, and
(b) a specific mapping of address bits to bus/device/function/
register
MCFG and _CBA assume both (a) and (b), so I think a device with
non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a
vendor-specific _HID or a device-specific method, e.g., _DSM, could
communicate that.
Jon, I agree that we should avoid describing non-standardized
hardware
Post by Bjorn Helgaas
in Linux-specific ways. Is there a mechanism in use already? How
does Windows handle this? DMI is a poor long-term solution because
it
Post by Bjorn Helgaas
requires ongoing maintenance for new platforms, but I think it's OK
for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs
can ship new platforms without requiring a new quirk or a new _HID to
be added to a driver.
There would still be the problem of config access before the
namespace
Post by Bjorn Helgaas
is available, i.e., the MCFG use case. I don't know how important
that is. Defining an MCFG extension seems like the most obvious
solution.
Your summary above is a perfect representation of the situation.
We had an opportunity to sync-up on the current status of ACPI PCI
for ARM64 (and talked about a way forward for this series, which
includes quirks handling), let me summarize it here for everyone
involved so that we can agree on a way forward.
1) ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG
ECAM for config space is basically ready (Tomasz and JC addressed
Rafael's concerns in relation to ARM64 specific code, and managed
to find a way to allocate domain numbers in preparation for Arnd
pci_create_root_bus() clean-up, v8 to be posted shortly and should
be final). This provides support for de-facto ACPI/PCI ECAM base
standard for ARM64 (with a clean-split between generic code and ARM64
bits, where ARM64, like X86 and IA64, manages in arch code IO space and
PCI resources, to be further consolidated in the near future).
I do not think anyone can complain about the generality of what we
achieved, for systems that are PCI standard (yes, PCI STANDARD) this
would just be sufficient.
2) In a real world (1) is not enough. Some ARM64 platforms, not entirely
ECAM compliant, already shipped with the corresponding firmware that
we can't update. HW has ECAM quirks and to work around it in the kernel
we put forward many solutions to the problem, it is time we found a
solution (when, of course, (1) is completed and upstream).
Using the MCFG table OEMID matching floated around in this thread
would work fine for most of the platforms (and cross-OS) that have
shipped with HW ECAM quirks, so I think that's the starting point for
our solution and that's how we can sort this out, _today_.
MCFG OEMID <-> PCI config space ops
3) (2) does not just work on some platforms (and we can't predict the
future either - actually I can, it is three letters, ECAM), simply
because MCFG OEMID matching does not provide a way to attach further
data to the MCFG (eg if config space for, say, bus 0 domain 0, is not
ECAM compliant, the config region can't be handled and must not be
handled through a corresponding MCFG region.
That's the problem Gabriele is facing and wants to solve through
https://lkml.org/lkml/2016/3/9/91
in the respective ACPI tables-bindings. It may be an idea worth
pursuing, it does not solve (2) simply because that FW has shipped,
we can't patch it any longer.
Hence to finally support ACPI PCI on ARM64 I suggest we carry out the
- Let's complete/merge (1), that's fundamental to this whole thread
- On top of (1) we apply a quirking mechanism based on (2) that allows
us to boot mainline with boxes shipping today with no FW update required.
- We devise a way to handle quirks that is more generic than (2) so that
can we can accomodate further platforms that can't rely on (2) but
have more leeway in terms of FW updates.
I hope that's a reasonable plan, Tomasz's v8 series coming to kick it off.
Thanks for summarizing.

100% agree on the summary and next steps.

Cheers

Gab
Thank you,
Lorenzo
Jon Masters
2016-05-24 04:30:01 UTC
Permalink
Post by Lorenzo Pieralisi
The only way this can be implemented is by pretending that the
ACPI/PCI arch/arm64 implementation is generic code (that's what this
series does), move it to /drivers (where it is in this series), and
implement _DSD vendor specific bindings (per HID) to set-up the pci
operations; whether this solution should go upstream, given that it
is just a short-term solution for early platforms bugs, it is another
story and my personal answer is no.
Just for completeness, let me also followup to this one.

We have real, shipping, systems in the field based on ARMv8. For
example, HPE Moonshot ProLiant m400. Not everyone loves the first
generation of anything (Applied get a lot of stick, but the reality is
that someone had to go first, and whoever that was was going to learn
all of the lessons that others don't need to) but it is out there and we
need an upstream kernel solution that includes support for that.

In the server world, we (speaking as a major distro vendor here) are not
going to entertain a situation in which non-upstream patches are needed
to even boot a platform. That simply won't do. We need to separate out
the issue of getting the core in place from the quirks, but then we need
quirks that include support for all early ARMv8 platforms that are out
there today. If we can't get to a point where a Moonshot[0] cartridge
boots out of the box with an upstream kernel, let's just give up and do
something else instead :) (joke)

Jon.

[0] HPE have been *amazingly* patient with this stuff. They've reworked
the firmware when someone (cough) pointed out that the early stuff
they'd been fed was not built according to the standards (U-Boot). They
have *really good* UEFI and ACPI enabled firmware that is running
RHEL(SA) great. But that's not good enough. We don't ship a distro with
hacks. We ship a distro derived from upstream code (although we might
have to backport a lot of stuff later). There's wiggle room, but there
is not wiggle room for core platforms. On ARM, users and developers
*will* be able to take an upstream kernel and boot it on their RHEL
install. And they *will* be able to reproduce problems against upstream,
and help develop against upstream, and further the upstream first
mentality in the ARM ecosystem. There will not be "oh well, it runs RHEL
so that's good enough for the early generation...".
--
Computer Architect | Sent from my Fedora powered laptop
Gabriele Paoloni
2016-05-20 08:20:01 UTC
Permalink
Hi Ard, Jon
-----Original Message-----
Sent: 20 May 2016 08:38
To: Jon Masters
Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host
controller
Post by Jon Masters
Hi Tomasz, all,
Post by Tomasz Nowicki
Post by Gabriele Paoloni
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it
decided to drop quirk handling?
I had such requests in previous series.
A quick note on quirk handling. This, I believe, applies post-merge
of
Post by Jon Masters
the base infrastructure, which I realize will likely not have quirks.
We've some "gen1" ARMv8 server platforms where we end up doing quirks
(for things like forcing 32-bit config space accessors and the like)
due
Post by Jon Masters
to people repurposing existing embedded PCIe IP blocks or using them
for
Post by Jon Masters
the first time (especially in servers), and those being involved in
the
Post by Jon Masters
design not necessarily seeing this problem ahead of time, or not
realizing that it would be an issue for servers. In the early days of
ARM server designs 3-4 years ago, many of us had never really played
with ECAM or realized how modern topologies are built.
Anyway. We missed this one in our SBSA requirements. They say (words
to
Post by Jon Masters
the effect of) "thou shalt do PCIe the way it is done on servers" but
they aren't prescriptive, and they don't tell people how that
actually
Post by Jon Masters
is in reality. That is being fixed. A lot of things are happening
behind
Post by Jon Masters
the scenes - especially with third party IP block providers (all of
whom
Post by Jon Masters
myself and others are speaking with directly about this) - to ensure
that the next wave of designs won't repeat these mistakes. We don't
have
Post by Jon Masters
a time machine, but we can contain this from becoming an ongoing mess
for upstream, and we will do so. It won't be a zoo.
Various proposals have arisen for how to handle quirks in the longer
term, including elaborate frameworks and tables to describe them
generically. I would like to caution against such approaches,
especially
Post by Jon Masters
in the case that they deviate from practice on x86, or prior to being
standardized fully with other Operating System vendors. I don't
expect
Post by Jon Masters
there to be too many more than the existing initial set of quirks we
have seen posted. A number of "future" server SoCs have already been
fixed prior to silicon, and new design starts are being warned not to
make this a problem for us to have to clean up later.
So, I would like to suggest that the eventual framework mirror the
existing approach on x86 systems (matching DMI, etc.) and not be made
into some kind of generic, utopia. This is a case where we want there
to
Post by Jon Masters
be pain involved (and upstream patches required) when people screw up,
so that they have a level of pain in response to ever making this
mistake in the future. If we try to create too grand a generic scheme
and make it too easy to handle this kind of situation beyond the
small
Post by Jon Masters
number of existing offenders, we undermine efforts to force vendors
to
Post by Jon Masters
ensure that their IP blocks are compliant going forward.
I understand that there is a desire from the RedHat side to mimic x86
as closely as possible, but I never saw any technical justification
for that. DMI contains strings that are visible to userland, and you
effectively lock those down to certain values just so that the kernel
can distinguish a broken PCIe root complex from a working one. Linux
on x86 had no choice, since the overwhelming majority of existing
hardware misrepresented itself as generic, and DMI was the only thing
available to actually distinguish these broken implementations from
one another. This does not mean we should allow and/or encourage this
first gen hardware to misrepresent non-compliant hardware as compliant
as well.
Since you are talking to all the people involved, how about you
convince them to put something in the ACPI tables that allows the
kernel to distinguish those non-standard PCIe implementations from
hardware that is really generic? This way, we can sidestep the quirks
debate entirely, since it will simply be a different device as far as
the kernel is concerned. This is no worse than a quirk from a
practical point of view, since an older OS will be equally unable to
run on newer hardware, but it is arguably more true to the standards
compliance you tend to preach about, especially since this small pool
of third party IP could potentially be identified directly rather than
based on some divination of the SoC we may or may not be running on. I
am also convinced that adding support for an additional HID() to the
ACPI ECAM driver with some special config space handling wired in is
an easier sell upstream than making the same ugly mess x86 has had to
make because they did not have any choice to begin with.
In our case (HiSilicon Hip05/Hip06) we are using the Designware IP
that unfortunately is non-ECAM for the RC config space.

A possible ACPI table solution was discussed already in this thread

https://lkml.org/lkml/2016/3/14/722

where <<Name (_CID, "PNP0C02") // Motherboard reserved resource>>
is used to specify an Host Controller specific resource.
It looks to me that this can be an approach that can accommodate
different vendors/scenarios and Bjorn seemed to be quite ok with
it.

It comes without saying that for future HW releases we all should
make an effort to deliver fully ECAM compliant controllers.

What's your view about this approach?

Thanks

Gab
If we do need a quirks handling mechanism, I still don't see how the
x86 situation extrapolates to ARM. ACPI offers plenty of ways for a
SoC vendor to identify the make and particular revision, and quirks
could be keyed off of that.
Jon Masters
2016-05-20 08:30:02 UTC
Permalink
Post by Gabriele Paoloni
Hi Ard, Jon
Hi Gabriele :)
Post by Gabriele Paoloni
In our case (HiSilicon Hip05/Hip06) we are using the Designware IP
that unfortunately is non-ECAM for the RC config space.
Yea, I know, and I've pinged them already.
Post by Gabriele Paoloni
A possible ACPI table solution was discussed already in this thread
https://lkml.org/lkml/2016/3/14/722
where <<Name (_CID, "PNP0C02") // Motherboard reserved resource>>
is used to specify an Host Controller specific resource.
It looks to me that this can be an approach that can accommodate
different vendors/scenarios and Bjorn seemed to be quite ok with
it.
Yeah, pondering that. We'll chat with a few others about it.
Post by Gabriele Paoloni
It comes without saying that for future HW releases we all should
make an effort to deliver fully ECAM compliant controllers.
Right. Like I said, a number of designs have been fixed already.
Post by Gabriele Paoloni
What's your view about this approach?
Will followup over the weekend.

Jon.
--
Computer Architect | Sent from my Fedora powered laptop
Duc Dang
2016-05-13 03:00:02 UTC
Permalink
From the functionality point of view this series may be split into the
1. New ECAM API and update for users of the pci-host-common API
2. Necessary fixes as the preparation for using driver on ARM64.
3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
4. Enable above driver on ARM64
This has been tested on Cavium ThunderX server. Any help in reviewing and
testing is very appreciated.
I tried this series on APM X-Gene platforms (with ECAM fix-up quirk
added) and PCIe works fine.

Regards,
Duc Dang.
v6 -> v7
- drop quirks handling
- changes for ACPI companion and domain number assignment approach
- implement arch pcibios_{add|remove}_bus and call acpi_pci_{add|remove}_bus from there
- cleanups around nomenclature
- use resources oriented API for ECAM
- fix for based address calculation before mapping ECAM region
- remove useless lock for MCFG lookup
- move MCFG stuff to separated file pci_mcfg.c
- drop MCFG entries caching
- rebase against 4.6-rc7
v5 -> v6
- drop idea of x86 MMCONFIG code refactoring
https://lkml.org/lkml/2016/4/11/907
git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
- integrate Sinan's fix for releasing IO resources, see patch [06/13]
- added ACPI support for ThunderX ECAM and PEM drivers
- rebase against 4.6-rc2
v4 -> v5
- drop MCFG refactoring group patches 1-6 from series v4 and integrate Jayachandran's patch
https://patchwork.ozlabs.org/patch/575525/
- rewrite PCI legacy IRQs allocation
- squash two patches 11 and 12 from series v4, fixed bisection issue
- changelog improvements
- rebase against 4.5-rc3
v3 -> v4
- drop Jiang's fix http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
- add Lorenzo's fix patch 19/24
- ACPI PCI bus domain number assigning cleanup
- change resource management, we now claim and reassign resources
- improvements for applying quirks
- drop Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html dependency
- rebase against 4.5-rc1
v2 -> v3
- fix legacy IRQ assigning and IO ports registration
- remove reference to arch specific companion device for ia64
- move ACPI PCI host controller driver to pci_root.c
- drop generic domain assignment for x86 and ia64 as I am not
able to run all necessary test variants
- drop patch which cleaned legacy IRQ assignment since it belongs to
https://patchwork.ozlabs.org/patch/557504/
- extend MCFG quirk code
- rebase against 4.4
v1 -> v2
- move non-arch specific piece of code to dirver/acpi/ directory
- fix IO resource handling
- introduce PCI config accessors quirks matching
- moved ACPI_COMPANION_SET to generic code
v1 - https://lkml.org/lkml/2015/10/27/504
v2 - https://lkml.org/lkml/2015/12/16/246
v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html
v4 - https://lkml.org/lkml/2016/2/4/646
v5 - https://lkml.org/lkml/2016/2/16/426
v6 - https://lkml.org/lkml/2016/4/15/594
PCI: Provide common functions for ECAM mapping
PCI: generic, thunder: update to use generic ECAM API
pci, of: Move the PCI I/O space management to PCI core code.
pci: Add new function to unmap IO resources.
acpi, pci: Support IO resources when parsing PCI host bridge
resources.
pci, acpi: Provide a way to assign bus domain number.
pci, acpi: Handle ACPI companion assignment.
pci, acpi: Support for ACPI based generic PCI host controller
arm64, pci, acpi: ACPI support for legacy IRQs parsing and
consolidation with DT code.
arm64, pci, acpi: Provide ACPI-specific prerequisites for PCI bus
enumeration.
arm64, pci, acpi: Start using ACPI based PCI host controller driver
for ARM64.
arch/arm64/Kconfig | 1 +
arch/arm64/kernel/pci.c | 34 +++-----
drivers/acpi/Kconfig | 8 ++
drivers/acpi/Makefile | 1 +
drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++
drivers/acpi/pci_root.c | 33 ++++++++
drivers/acpi/pci_root_generic.c | 149 +++++++++++++++++++++++++++++++++
drivers/of/address.c | 116 +-------------------------
drivers/pci/Kconfig | 3 +
drivers/pci/Makefile | 2 +
drivers/pci/ecam.c | 161 ++++++++++++++++++++++++++++++++++++
drivers/pci/ecam.h | 72 ++++++++++++++++
drivers/pci/host/Kconfig | 1 +
drivers/pci/host/pci-host-common.c | 114 +++++++++++--------------
drivers/pci/host/pci-host-common.h | 47 -----------
drivers/pci/host/pci-host-generic.c | 52 +++---------
drivers/pci/host/pci-thunder-ecam.c | 39 ++-------
drivers/pci/host/pci-thunder-pem.c | 92 ++++++++++-----------
drivers/pci/pci.c | 150 ++++++++++++++++++++++++++++++++-
drivers/pci/probe.c | 2 +
include/linux/of_address.h | 9 --
include/linux/pci-acpi.h | 14 ++++
include/linux/pci.h | 11 ++-
23 files changed, 823 insertions(+), 385 deletions(-)
create mode 100644 drivers/acpi/pci_mcfg.c
create mode 100644 drivers/acpi/pci_root_generic.c
create mode 100644 drivers/pci/ecam.c
create mode 100644 drivers/pci/ecam.h
delete mode 100644 drivers/pci/host/pci-host-common.h
--
1.9.1
Jeremy Linton
2016-05-19 18:20:01 UTC
Permalink
From the functionality point of view this series may be split into the
1. New ECAM API and update for users of the pci-host-common API
2. Necessary fixes as the preparation for using driver on ARM64.
3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
4. Enable above driver on ARM64
Thomasz,

Unless i'm doing something wrong, this patch causes a build break if
NUMA is enabled in a 4.6+ kernel.

Thanks,



drivers/built-in.o: In function `pci_device_add':
/home/jlinton/linux-main/drivers/pci/probe.c:1744: undefined reference
to `pcibus_to_node'
drivers/built-in.o: In function `pci_create_root_bus':
/home/jlinton/linux-main/drivers/pci/probe.c:2163: undefined reference
to `pcibus_to_node'
drivers/built-in.o: In function `cpulistaffinity_show':
/home/jlinton/linux-main/drivers/pci/pci-sysfs.c:123: undefined
reference to `pcibus_to_node'
/home/jlinton/linux-main/drivers/pci/pci-sysfs.c:123: undefined
reference to `pcibus_to_node'
drivers/built-in.o: In function `cpuaffinity_show':
/home/jlinton/linux-main/drivers/pci/pci-sysfs.c:114: undefined
reference to `pcibus_to_node'
drivers/built-in.o:/home/jlinton/linux-main/drivers/pci/pci-sysfs.c:114:
more undefined references to `pcibus_to_node' follow
Makefile:937: recipe for target 'vmlinux' failed
Dongdong Liu
2016-05-23 11:40:01 UTC
Permalink
From the functionality point of view this series may be split into the
1. New ECAM API and update for users of the pci-host-common API
2. Necessary fixes as the preparation for using driver on ARM64.
3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
4. Enable above driver on ARM64
This has been tested on Cavium ThunderX server. Any help in reviewing and
testing is very appreciated.
Based on the patchset (with ECAM fix-up quirk added) and added the patch(Add ACPI support for HiSilicon PCIe Host Controllers).
Tested on the HiSilicon ARM64 D02 board.
It can work ok with Intel 82599 networking card.
This is the bootup log which contains PCIe host and Intel 82599 networking card part.

Tested-by: Dongdong Liu <***@huawei.com>

EFI stub: Booting Linux Kernel...
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...
GMAC ExitBootServicesEvent
SMMU ExitBootServicesEvent
[ 0.000000] Booting Linux on physical CPU 0x20000
[ 0.000000] Linux version 4.6.0-rc1+ (***@linux-ioko) (gcc version 4.9.3 20150211 (prerelease) (20150316) ) #43 SMP PREEMPT Thu May 19 17:15:30 CST 2016
[ 0.000000] Boot CPU: AArch64 Processor [411fd071]
[ 0.000000] earlycon: uart8250 at MMIO32 0x0000000080300000 (options '')
[ 0.000000] bootconsole [uart8250] enabled
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] EFI v2.50 by EDK II
[ 0.000000] efi: SMBIOS=0x7a650000 SMBIOS 3.0=0x7a630000 ACPI=0x7aba0000 ACPI 2.0=0x7aba0014
[ 0.000000] cma: Reserved 16 MiB at 0x000000007e800000
[ 0.000000] ACPI: Early table checksum verification disabled
[ 0.000000] ACPI: RSDP 0x000000007ABA0014 000024 (v02 HISI )
[ 0.000000] ACPI: XSDT 0x000000007A7000E8 000064 (v01 HISI HISI-D02 20140727 01000013)
[ 0.000000] ACPI: FACP 0x000000007A5F0000 00010C (v05 HISI HISI-D02 20140727 HISI 00000099)
[ 0.000000] ACPI: DSDT 0x000000007A5A0000 001656 (v01 HISI HISI-D02 20140727 INTL 20150619)
[ 0.000000] ACPI: DBG2 0x000000007A610000 00005A (v00 HISI HISI-D02 20140727 HISI 00000099)
[ 0.000000] ACPI: GTDT 0x000000007A5E0000 000060 (v02 HISI HISI-D02 20140727 HISI 00000099)
[ 0.000000] ACPI: APIC 0x000000007A5D0000 000554 (v01 HISI HISI-D02 20140727 HISI 00000099)
[ 0.000000] ACPI: MCFG 0x000000007A5C0000 00004C (v01 HISI HISI-D02 20140727 HISI 00000099)
[ 0.000000] ACPI: SPCR 0x000000007A5B0000 000050 (v02 HISI HISI-D02 20140727 HISI 00000099)
[ 0.000000] ACPI: IORT 0x000000007A590000 0001FC (v00 INTEL TEMPLATE 00000000 INTL 20150619)
[ 0.000000] ACPI: SSDT 0x000000007A580000 00046E (v01 HISI SAS0 20140727 INTL 20150619)
[ 0.000000] ACPI: SPCR: console: uart,mmio,0x80300000,115200
[ 0.000000] psci: probing for conduit method from ACPI.
NOTICE: [psci_smc_handler]:[347L] PSCI_VERSION CALL
NOTICE: [psci_version]:[99L] PSCI_MAJOR_VER: 10000: PSCI_MINOR_VER: 0

0808?844
[ 0.000000] psci: PSCIv1.0 detected in firmware.
[ 0.000000] psci: Using standard PSCI v0.2 function IDs

0808?844
[ 0.000000] psci: MIGRATE_INFO_TYPE not supported.

0808?844

0808?844
[ 0.000000] percpu: Embedded 20 pages/cpu @ffffffd1ffe7e000 s43008 r8192 d30720 u81920
[ 0.000000] Detected PIPT I-cache on CPU0
[ 0.000000] CPU features: enabling workaround for ARM erratum 832075
[ 0.000000] CPU features: enabling workaround for ARM erratum 834220
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 2063376
[ 0.000000] Kernel command line: console=ttyS0,115200 earlycon=uart8250,mmio32,0x80300000 initrd=filesystem.cpio.gz acpi=force pcie_aspm=off
[ 0.000000] PCIe ASPM is disabled
[ 0.000000] log_buf_len individual max cpu contribution: 4096 bytes
[ 0.000000] log_buf_len total cpu_extra contributions: 61440 bytes
[ 0.000000] log_buf_len min size: 16384 bytes
[ 0.000000] log_buf_len: 131072 bytes
[ 0.000000] early log buf free: 12988(79%)
[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[ 0.000000] software IO TLB [mem 0x764e0000-0x7a4e0000] (64MB) mapped at [ffffffc0764e0000-ffffffc07a4dffff]
[ 0.000000] Memory: 8110296K/8384512K available (7240K kernel code, 632K rwdata, 3028K rodata, 840K init, 247K bss, 257832K reserved, 16384K cma-reserved)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] modules : 0xffffff8000000000 - 0xffffff8008000000 ( 128 MB)
[ 0.000000] vmalloc : 0xffffff8008000000 - 0xffffffbdbfff0000 ( 246 GB)
[ 0.000000] .text : 0xffffff8008080000 - 0xffffff8008790000 ( 7232 KB)
[ 0.000000] .rodata : 0xffffff8008790000 - 0xffffff8008a89000 ( 3044 KB)
[ 0.000000] .init : 0xffffff8008a89000 - 0xffffff8008b5b000 ( 840 KB)
[ 0.000000] .data : 0xffffff8008b5b000 - 0xffffff8008bf9200 ( 633 KB)
[ 0.000000] vmemmap : 0xffffffbdc0000000 - 0xffffffbfc0000000 ( 8 GB maximum)
[ 0.000000] 0xffffffbdc0000000 - 0xffffffbe08000000 ( 1152 MB actual)
[ 0.000000] fixed : 0xffffffbffe7fd000 - 0xffffffbffec00000 ( 4108 KB)
[ 0.000000] PCI I/O : 0xffffffbffee00000 - 0xffffffbfffe00000 ( 16 MB)
[ 0.000000] memory : 0xffffffc000000000 - 0xffffffd200000000 ( 73728 MB)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=16, Nodes=1
[ 0.000000] Preemptible hierarchical RCU implementation.
[ 0.000000] Build-time adjustment of leaf fanout to 64.
[ 0.000000] RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=16.
[ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=16
[ 0.000000] NR_IRQS:64 nr_irqs:64 0
[ 0.000000] GIC: Using split EOI/Deactivate mode
[ 0.000000] ***@0x8c000000
[ 0.000000] ITS: allocated 65536 Devices @11f6c80000 (psz 4K, shr 1)
[ 0.000000] ITS: allocated 512 Virtual CPUs @11f6c0f000 (psz 4K, shr 1)
[ 0.000000] ITS: allocated 512 Interrupt Collections @11f6c20000 (psz 4K, shr 1)
[ 0.000000] ***@0xc6000000
[ 0.000000] ITS: allocated 65536 Devices @11f6d00000 (psz 4K, shr 1)
[ 0.000000] ITS: allocated 512 Virtual CPUs @11f6c21000 (psz 4K, shr 1)
[ 0.000000] ITS: allocated 512 Interrupt Collections @11f6c22000 (psz 4K, shr 1)
[ 0.000000] ***@0xa3000000
[ 0.000000] ITS: allocated 65536 Devices @11f6d80000 (psz 4K, shr 1)
[ 0.000000] ITS: allocated 512 Virtual CPUs @11f6c24000 (psz 4K, shr 1)
[ 0.000000] ITS: allocated 512 Interrupt Collections @11f6c25000 (psz 4K, shr 1)
[ 0.000000] ***@0xb7000000
[ 0.000000] ITS: allocated 65536 Devices @11f6e00000 (psz 4K, shr 1)
[ 0.000000] ITS: allocated 512 Virtual CPUs @11f6c26000 (psz 4K, shr 1)
[ 0.000000] ITS: allocated 512 Interrupt Collections @11f6c27000 (psz 4K, shr 1)
[ 0.000000] GIC: using LPI property table @0x00000011f6c60000
[ 0.000000] ITS: Allocated 1792 chunks for LPIs
[ 0.000000] CPU0: found redistributor 20000 region 0:0x000000008d100000
[ 0.000000] CPU0: using LPI pending table @0x00000011f6c70000
[ 0.000000] Unable to get hardware information used for virtualization
[ 0.000000] GTDT: No Platform Timer structures.
[ 0.000000] arch_timer: Can't find GT Block.
[ 0.000000] Architected cp15 and mmio timer(s) running at 50.00MHz (phys/phys).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
[ 0.000001] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns
[ 0.008025] Console: colour dummy device 80x25
[ 0.012459] Calibrating delay loop (skipped), value calculated using timer frequency.. 100.00 BogoMIPS (lpj=200000)
[ 0.022850] pid_max: default: 32768 minimum: 301
[ 0.027450] ACPI: Core revision 20160108
[ 0.032284] ACPI: 2 ACPI AML tables successfully acquired and loaded
[ 0.038617]
[ 0.040126] Security Framework initialized
[ 0.044213] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes)
[ 0.051053] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes)
[ 0.058747] ASID allocator initialised with 65536 entries
[ 0.064393] PCI/MSI: ***@0x8c000000 domain created
[ 0.069162] PCI/MSI: ***@0xc6000000 domain created
[ 0.073927] PCI/MSI: ***@0xa3000000 domain created
[ 0.078693] PCI/MSI: ***@0xb7000000 domain created
[ 0.083463] Platform MSI: ***@000000008c000000 domain created
[ 0.089525] Platform MSI: ***@00000000c6000000 domain created
[ 0.095586] Platform MSI: ***@00000000a3000000 domain created
[ 0.101648] Platform MSI: ***@00000000b7000000 domain created
[ 0.107753] Remapping and enabling EFI services.
[ 0.112370] EFI remap 0x000000007a4e0000 => 0000000020000000
[ 0.118185] EFI remap 0x000000007a530000 => 0000000020050000
[ 0.124008] EFI remap 0x000000007a620000 => 00000000200a0000
[ 0.129822] EFI remap 0x000000007a6b0000 => 0000000020130000
[ 0.135636] EFI remap 0x000000007a710000 => 0000000020180000
[ 0.141454] EFI remap 0x000000007a760000 => 00000000201d0000
[ 0.147269] EFI remap 0x000000007a7b0000 => 0000000020220000
[ 0.153083] EFI remap 0x000000007a800000 => 0000000020270000
[ 0.158897] EFI remap 0x000000007a850000 => 00000000202c0000
[ 0.164711] EFI remap 0x000000007a8a0000 => 0000000020310000
[ 0.170525] EFI remap 0x000000007a8f0000 => 0000000020360000
[ 0.176339] EFI remap 0x000000007a940000 => 00000000203b0000
[ 0.182160] EFI remap 0x000000007a990000 => 0000000020400000
[ 0.187974] EFI remap 0x000000007aa00000 => 0000000020470000
[ 0.193788] EFI remap 0x000000007aa50000 => 00000000204c0000
[ 0.199602] EFI remap 0x000000007aaa0000 => 0000000020510000
[ 0.205417] EFI remap 0x000000007aaf0000 => 0000000020560000
[ 0.211230] EFI remap 0x000000007ab40000 => 00000000205b0000
[ 0.217043] EFI remap 0x000000007fbb0000 => 0000000020600000
[ 0.222849] EFI remap 0x0000000080300000 => 0000000020630000
[ 0.228654] EFI remap 0x00000000a00f0000 => 0000000020640000
[ 0.234457] EFI remap 0x00000000a4000000 => 0000000020800000
[ 0.240266] EFI remap 0x00000000a6000000 => 0000000021800000
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20001 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x1

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x1


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c080
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d190
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20002 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x1

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x1


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c100
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d3a0
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20003 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x1

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x1


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c180
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d5b0
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20100 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x1

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x3


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c200
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d7c0
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20101 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x3

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x3


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c280
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d9d0
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20102 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x3

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x3


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c300
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3dbe0
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20103 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x3

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x3


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c380
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ddf0
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20200 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x3

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x7


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c400
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e000
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20201 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x7

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x7


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c480
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e210
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20202 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x7

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x7


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c500
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e420
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20203 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x7

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x7


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c580
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e630
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20300 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x7

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0xf


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c600
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e840
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20301 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0xf

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0xf


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c680
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ea50
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20302 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0xf

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0xf


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c700
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ec60
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL
NOTICE: [psci_smc_handler]:[409L] x1=0x20303 x2=0x82870 x3=0x0
NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0xf

NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0xf


0808?8AB44
NOTICE: [psci_afflvl_power_on_finish]:[504L]
NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c780
NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ee70
NOTICE: [psci_afflvl_power_on_finish]:[562L]

00082870
[ 0.288031] Detected PIPT I-cache on CPU1
[ 0.288043] CPU1: found redistributor 20001 region 1:0x000000008d130000
[ 0.288064] CPU1: using LPI pending table @0x00000011f6410000
[ 0.288122] CPU1: Booted secondary processor [411fd071]
[ 0.331143] Detected PIPT I-cache on CPU2
[ 0.331150] CPU2: found redistributor 20002 region 2:0x000000008d160000
[ 0.331170] CPU2: using LPI pending table @0x00000011f6440000
[ 0.331218] CPU2: Booted secondary processor [411fd071]
[ 0.374257] Detected PIPT I-cache on CPU3
[ 0.374263] CPU3: found redistributor 20003 region 3:0x000000008d190000
[ 0.374283] CPU3: using LPI pending table @0x00000011f6480000
[ 0.374328] CPU3: Booted secondary processor [411fd071]
[ 0.417372] Detected PIPT I-cache on CPU4
[ 0.417380] CPU4: found redistributor 20100 region 4:0x000000008d1c0000
[ 0.417400] CPU4: using LPI pending table @0x00000011f64c0000
[ 0.417447] CPU4: Booted secondary processor [411fd071]
[ 0.460484] Detected PIPT I-cache on CPU5
[ 0.460491] CPU5: found redistributor 20101 region 5:0x000000008d1f0000
[ 0.460511] CPU5: using LPI pending table @0x00000011f64f0000
[ 0.460558] CPU5: Booted secondary processor [411fd071]
[ 0.503598] Detected PIPT I-cache on CPU6
[ 0.503605] CPU6: found redistributor 20102 region 6:0x000000008d220000
[ 0.503625] CPU6: using LPI pending table @0x00000011f6530000
[ 0.503669] CPU6: Booted secondary processor [411fd071]
[ 0.546711] Detected PIPT I-cache on CPU7
[ 0.546718] CPU7: found redistributor 20103 region 7:0x000000008d250000
[ 0.546738] CPU7: using LPI pending table @0x00000011f6560000
[ 0.546783] CPU7: Booted secondary processor [411fd071]
[ 0.589826] Detected PIPT I-cache on CPU8
[ 0.589835] CPU8: found redistributor 20200 region 8:0x000000008d280000
[ 0.589857] CPU8: using LPI pending table @0x00000011f65a0000
[ 0.589910] CPU8: Booted secondary processor [411fd071]
[ 0.632939] Detected PIPT I-cache on CPU9
[ 0.632946] CPU9: found redistributor 20201 region 9:0x000000008d2b0000
[ 0.632967] CPU9: using LPI pending table @0x00000011f65e0000
[ 0.633014] CPU9: Booted secondary processor [411fd071]
[ 0.676052] Detected PIPT I-cache on CPU10
[ 0.676060] CPU10: found redistributor 20202 region 10:0x000000008d2e0000
[ 0.676081] CPU10: using LPI pending table @0x00000011f6610000
[ 0.676126] CPU10: Booted secondary processor [411fd071]
[ 0.719166] Detected PIPT I-cache on CPU11
[ 0.719173] CPU11: found redistributor 20203 region 11:0x000000008d310000
[ 0.719195] CPU11: using LPI pending table @0x00000011f6650000
[ 0.719240] CPU11: Booted secondary processor [411fd071]
[ 0.762280] Detected PIPT I-cache on CPU12
[ 0.762289] CPU12: found redistributor 20300 region 12:0x000000008d340000
[ 0.762311] CPU12: using LPI pending table @0x00000011f6680000
[ 0.762360] CPU12: Booted secondary processor [411fd071]
[ 0.805392] Detected PIPT I-cache on CPU13
[ 0.805400] CPU13: found redistributor 20301 region 13:0x000000008d370000
[ 0.805421] CPU13: using LPI pending table @0x00000011f66c0000
[ 0.805466] CPU13: Booted secondary processor [411fd071]
[ 0.848506] Detected PIPT I-cache on CPU14
[ 0.848513] CPU14: found redistributor 20302 region 14:0x000000008d3a0000
[ 0.848534] CPU14: using LPI pending table @0x00000011f6700000
[ 0.848579] CPU14: Booted secondary processor [411fd071]
[ 0.891620] Detected PIPT I-cache on CPU15
[ 0.891628] CPU15: found redistributor 20303 region 15:0x000000008d3d0000
[ 0.891648] CPU15: using LPI pending table @0x00000011f6750000
[ 0.891693] CPU15: Booted secondary processor [411fd071]
[ 0.891723] Brought up 16 CPUs
[ 1.220754] SMP: Total of 16 processors activated.
[ 1.225520] CPU features: detected feature: GIC system register CPU interface
[ 1.232620] CPU: All CPU(s) started at EL2
[ 1.236718] alternatives: patching kernel code
[ 1.243905] devtmpfs: initialized
[ 1.247446] SMBIOS 3.0.0 present.
[ 1.250851] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 1.260790] pinctrl core: initialized pinctrl subsystem
[ 1.266365] NET: Registered protocol family 16
[ 1.282800] cpuidle: using governor menu
[ 1.286747] vdso: 2 pages (1 code @ ffffff8008796000, 1 data @ ffffff8008b60000)
[ 1.294122] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[ 1.301211] DMA: preallocated 256 KiB pool for atomic allocations
[ 1.307340] ACPI: bus type PCI registered
[ 1.311388] Serial: AMBA PL011 UART driver
[ 1.331638] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 1.338373] ACPI: Added _OSI(Module Device)
[ 1.342562] ACPI: Added _OSI(Processor Device)
[ 1.346984] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 1.351664] ACPI: Added _OSI(Processor Aggregator Device)
[ 1.358469] ACPI: Interpreter enabled
[ 1.362116] ACPI: Using GIC for interrupt routing
[ 1.366815] ACPI: MCFG table loaded, 2 entries detected
[ 1.375180] Hisilicon MBIGEN-V1 HISI0151:00: Allocated 256 MSIs
[ 1.381158] Hisilicon MBIGEN-V1 HISI0151:01: Allocated 640 MSIs
[ 1.387123] Hisilicon MBIGEN-V1 HISI0151:02: Allocated 256 MSIs
[ 1.393091] Hisilicon MBIGEN-V1 HISI0151:03: Allocated 640 MSIs
[ 1.399213] ACPI: IORT: can't find node related to (null) device
[ 1.405292] ACPI: IORT: can't find node related to (null) device
[ 1.411324] ACPI: IORT: can't find node related to (null) device
[ 1.417348] ACPI: IORT: can't find node related to (null) device
[ 1.423370] ACPI: IORT: can't find node related to (null) device
[ 1.429391] ACPI: IORT: can't find node related to (null) device
[ 1.436634] ACPI: IORT: can't find node related to (null) device
[ 1.442681] ACPI: IORT: can't find node related to (null) device
[ 1.448698] ACPI: IORT: can't find node related to (null) device
[ 1.454712] ACPI: IORT: can't find node related to (null) device
[ 1.460724] ACPI: IORT: can't find node related to (null) device
[ 1.466764] ACPI: PCI Root Bridge [PCI1] (domain 0001 [bus 40-7f])
[ 1.472918] acpi HISI0080:00: _OSC: OS supports [ExtendedConfig Segments MSI]
[ 1.480025] acpi HISI0080:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 1.486771] acpi HISI0080:00: ECAM at [mem 0x22004000000-0x22007ffffff] for [bus 40-7f]
[ 1.494762] Remapped I/O 0x000002200fff0000 to [io 0x0000-0xffff window]
[ 1.501592] PCI host bridge to bus 0001:40
[ 1.505671] pci_bus 0001:40: root bus resource [mem 0x22008000000-0x2200ffeffff window] (bus address [0xb0000000-0xb7feffff])
[ 1.516919] pci_bus 0001:40: root bus resource [io 0x0000-0xffff window]
[ 1.523675] pci_bus 0001:40: root bus resource [bus 40-7f]
[ 1.534090] pci 0001:41:00.0: VF(n) BAR0 space: [mem 0x22008e08000-0x22008f07fff 64bit pref] (contains BAR0 for 64 VFs)
[ 1.545323] pci 0001:41:00.0: VF(n) BAR3 space: [mem 0x22008f08000-0x22009007fff 64bit pref] (contains BAR3 for 64 VFs)
[ 1.568223] pci 0001:41:00.1: VF(n) BAR0 space: [mem 0x22008c04000-0x22008d03fff 64bit pref] (contains BAR0 for 64 VFs)
[ 1.579460] pci 0001:41:00.1: VF(n) BAR3 space: [mem 0x22008d04000-0x22008e03fff 64bit pref] (contains BAR3 for 64 VFs)
[ 1.597823] pci 0001:40:00.0: BAR 15: assigned [mem 0x22008000000-0x220095fffff pref]
[ 1.605616] pci 0001:40:00.0: BAR 13: assigned [io 0x1000-0x1fff]
[ 1.611771] pci 0001:41:00.0: BAR 0: assigned [mem 0x22008000000-0x220083fffff 64bit pref]
[ 1.620245] pci 0001:41:00.0: BAR 6: assigned [mem 0x22008400000-0x220087fffff pref]
[ 1.627952] pci 0001:41:00.1: BAR 0: assigned [mem 0x22008800000-0x22008bfffff 64bit pref]
[ 1.636430] pci 0001:41:00.1: BAR 6: assigned [mem 0x22008c00000-0x22008ffffff pref]
[ 1.644138] pci 0001:41:00.0: BAR 4: assigned [mem 0x22009000000-0x22009003fff 64bit pref]
[ 1.652610] pci 0001:41:00.0: BAR 7: assigned [mem 0x22009004000-0x22009103fff 64bit pref]
[ 1.661083] pci 0001:41:00.0: BAR 10: assigned [mem 0x22009104000-0x22009203fff 64bit pref]
[ 1.669640] pci 0001:41:00.1: BAR 4: assigned [mem 0x22009204000-0x22009207fff 64bit pref]
[ 1.678115] pci 0001:41:00.1: BAR 7: assigned [mem 0x22009208000-0x22009307fff 64bit pref]
[ 1.686591] pci 0001:41:00.1: BAR 10: assigned [mem 0x22009308000-0x22009407fff 64bit pref]
[ 1.695148] pci 0001:41:00.0: BAR 2: assigned [io 0x1000-0x101f]
[ 1.701285] pci 0001:41:00.1: BAR 2: assigned [io 0x1020-0x103f]
[ 1.707421] pci 0001:40:00.0: PCI bridge to [bus 41-42]
[ 1.712621] pci 0001:40:00.0: bridge window [io 0x1000-0x1fff]
[ 1.718685] pci 0001:40:00.0: bridge window [mem 0x22008000000-0x220095fffff pref]
[ 1.726434] ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 80-bf])
[ 1.732594] acpi HISI0080:01: _OSC: OS supports [ExtendedConfig Segments MSI]
[ 1.739696] acpi HISI0080:01: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 1.746425] acpi HISI0080:01: link status is down
[ 1.751106] acpi HISI0080:01: ECAM at [mem 0x24008000000-0x2400bffffff] for [bus 80-bf]
[ 1.759090] Remapped I/O 0x000002400fff0000 to [io 0x10000-0x1ffff window]
[ 1.766086] PCI host bridge to bus 0002:80
[ 1.770172] pci_bus 0002:80: root bus resource [mem 0x2400c000000-0x2400ffeffff window] (bus address [0xc0000000-0xc3feffff])
[ 1.781422] pci_bus 0002:80: root bus resource [io 0x10000-0x1ffff window] (bus address [0x0000-0xffff])
[ 1.790940] pci_bus 0002:80: root bus resource [bus 80-bf]
[ 1.796414] pci 0002:80:00.0: ignoring class 0x000000 (doesn't match header type 01)
[ 1.804311] pci 0002:80:00.0: not setting up bridge for bus 0002:81
[ 1.810956] ACPI: IORT: can't find node related to (null) device
[ 1.817228] ACPI: IORT: can't find node related to (null) device
[ 1.823453] vgaarb: loaded
[ 1.826246] SCSI subsystem initialized
[ 1.830103] ACPI: bus type USB registered
[ 1.834130] usbcore: registered new interface driver usbfs
[ 1.839606] usbcore: registered new interface driver hub
[ 1.844933] usbcore: registered new device driver usb
[ 1.850014] pps_core: LinuxPPS API ver. 1 registered
[ 1.854953] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <***@linux.it>
[ 1.864048] PTP clock support registered
[ 1.868069] Advanced Linux Sound Architecture Driver Initialized.
[ 1.874376] clocksource: Switched to clocksource arch_sys_counter
[ 1.880489] VFS: Disk quotas dquot_6.6.0
[ 1.884414] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 1.891385] pnp: PnP ACPI init
[ 1.894570] system 00:00: [mem 0xb0080000-0xb008ffff] has been reserved
[ 1.901209] system 00:01: [mem 0xb0090000-0xb009ffff] has been reserved
[ 1.907813] pnp: PnP ACPI: found 2 devices
[ 1.914467] NET: Registered protocol family 2
[ 1.919030] TCP established hash table entries: 65536 (order: 7, 524288 bytes)
[ 1.926352] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[ 1.933333] TCP: Hash tables configured (established 65536 bind 65536)
[ 1.939876] UDP hash table entries: 4096 (order: 5, 131072 bytes)
[ 1.945968] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes)
[ 1.952576] NET: Registered protocol family 1
[ 1.957013] RPC: Registered named UNIX socket transport module.
[ 1.962905] RPC: Registered udp transport module.
[ 1.967586] RPC: Registered tcp transport module.
[ 1.972266] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 1.978872] Unpacking initramfs...
[ 2.340835] Freeing initrd memory: 27492K (ffffffc01e520000 - ffffffc01fff9000)
[ 2.348645] kvm [1]: 8-bit VMID
[ 2.351776] kvm [1]: Hyp mode initialized successfully
[ 2.356892] kvm [1]: error: KVM vGIC probing failed
[ 2.361790] kvm [1]: virtual timer IRQ3
[ 2.366650] ACPI: IORT: can't find node related to (null) device
[ 2.373057] futex hash table entries: 4096 (order: 7, 524288 bytes)
[ 2.379403] audit: initializing netlink subsys (disabled)
[ 2.384813] audit: type=2000 audit(1.864:1): initialized
[ 2.390302] workingset: timestamp_bits=44 max_order=21 bucket_order=0
[ 2.400222] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 2.406342] NFS: Registering the id_resolver key type
[ 2.411393] Key type id_resolver registered
[ 2.415557] Key type id_legacy registered
[ 2.419609] fuse init (API version 7.24)
[ 2.423687] 9p: Installing v9fs 9p2000 file system support
[ 2.429982] io scheduler noop registered
[ 2.433946] io scheduler cfq registered (default)
[ 2.439027] pcieport 0001:40:00.0: can't derive routing for PCI INT A
[ 2.445438] pcieport 0001:40:00.0: PCI INT A: no GSI
[ 2.451126] xenfs: not registering filesystem on non-xen platform
[ 2.458262] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[ 2.465049] console [ttyS0] disabled
[ 2.468627] APMC0D08:00: ttyS0 at MMIO 0x80300000 (irq = 5, base_baud = 12500000) is a 16550A
[ 2.477133] console [ttyS0] enabled
[ 2.477133] console [ttyS0] enabled
[ 2.484137] bootconsole [uart8250] disabled
[ 2.484137] bootconsole [uart8250] disabled
[ 2.492728] SuperH (H)SCI(F) driver initialized
[ 2.497322] msm_serial: driver initialized
[ 2.501649] Failed to find cpu0 device node
[ 2.505852] Unable to detect cache hierarchy from DT for CPU 0
[ 2.514093] loop: module loaded
[ 2.517694] tun: Universal TUN/TAP device driver, 1.6
[ 2.522771] tun: (C) 1999-2004 Max Krasnyansky <***@qualcomm.com>
[ 2.529079] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[ 2.534939] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[ 2.540908] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.3.0-k
[ 2.547900] igb: Copyright (c) 2007-2014 Intel Corporation.
[ 2.553518] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.0.2-k
[ 2.561382] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[ 2.567350] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 4.2.1-k
[ 2.575039] ixgbe: Copyright (c) 1999-2015 Intel Corporation.
[ 2.580854] pcieport 0001:40:00.0: can't derive routing for PCI INT A
[ 2.587326] ixgbe 0001:41:00.0: PCI INT A: no GSI
[ 2.592178] ixgbe 0001:41:00.0: enabling device (0000 -> 0002)
[ 2.753716] ixgbe 0001:41:00.0: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16
[ 2.762393] ixgbe 0001:41:00.0: PCI Express bandwidth of 32GT/s available
[ 2.769213] ixgbe 0001:41:00.0: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
[ 2.776457] ixgbe 0001:41:00.0: MAC: 2, PHY: 17, SFP+: 5, PBA No: FFFFFF-0FF
[ 2.783537] ixgbe 0001:41:00.0: 68:a8:28:2e:c9:10
[ 2.792732] ixgbe 0001:41:00.0: Intel(R) 10 Gigabit Network Connection
[ 2.799312] pcieport 0001:40:00.0: can't derive routing for PCI INT B
[ 2.805783] ixgbe 0001:41:00.1: PCI INT B: no GSI
[ 2.810598] ixgbe 0001:41:00.1: enabling device (0000 -> 0002)
[ 3.949697] ixgbe 0001:41:00.1: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16
[ 3.958365] ixgbe 0001:41:00.1: PCI Express bandwidth of 32GT/s available
[ 3.965185] ixgbe 0001:41:00.1: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
[ 3.972427] ixgbe 0001:41:00.1: MAC: 2, PHY: 1, PBA No: FFFFFF-0FF
[ 3.978634] ixgbe 0001:41:00.1: 68:a8:28:2e:c9:11
[ 3.987790] ixgbe 0001:41:00.1: Intel(R) 10 Gigabit Network Connection
[ 3.994388] sky2: driver version 1.30
[ 3.998201] VFIO - User Level meta-driver version: 0.3
[ 4.003889] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 4.010459] ehci-pci: EHCI PCI platform driver
[ 4.014944] ehci-platform: EHCI generic platform driver
[ 4.020226] ehci-platform PNP0D20:00: EHCI Host Controller
[ 4.025745] ehci-platform PNP0D20:00: new USB bus registered, assigned bus number 1
[ 4.033572] ehci-platform PNP0D20:00: irq 6, io mem 0xa1000000
[ 4.050387] ehci-platform PNP0D20:00: USB 2.0 started, EHCI 1.00
[ 4.056652] hub 1-0:1.0: USB hub found
[ 4.060428] hub 1-0:1.0: 1 port detected
[ 4.064517] ehci-msm: Qualcomm On-Chip EHCI Host Controller
[ 4.070137] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 4.076351] ohci-pci: OHCI PCI platform driver
[ 4.080834] ohci-platform: OHCI generic platform driver
[ 4.086166] usbcore: registered new interface driver usb-storage
[ 4.092427] mousedev: PS/2 mouse device common for all mice
[ 4.161533] rtc-efi rtc-efi: rtc core: registered rtc-efi as rtc0
[ 4.168513] i2c /dev entries driver
[ 4.172251] sdhci: Secure Digital Host Controller Interface driver
[ 4.178458] sdhci: Copyright(c) Pierre Ossman
[ 4.182850] Synopsys Designware Multimedia Card Interface Driver
[ 4.188933] sdhci-pltfm: SDHCI platform and OF driver helper
[ 4.194701] ledtrig-cpu: registered to indicate activity on CPUs
[ 4.200950] usbcore: registered new interface driver usbhid
[ 4.206549] usbhid: USB HID core driver
[ 4.210569] ACPI: IORT: can't find node related to (null) device
[ 4.216837] NET: Registered protocol family 17
[ 4.221334] 9pnet: Installing 9P2000 support
[ 4.225652] Key type dns_resolver registered
[ 4.230166] registered taskstats version 1
[ 4.297850] rtc-efi rtc-efi: hctosys: unable to read the hardware clock
[ 4.304576] ALSA device list:
[ 4.307561] No soundcards found.
[ 4.311063] ttyS0 - failed to request DMA
[ 4.315348] Freeing unused kernel memory: 840K (ffffff8008a89000 - ffffff8008b5b000)
root@(none)$ ifconfig eth0 192.168.20.188
[ 15.679957] ixgbe 0001:41:00.0: registered PHC device on eth0
root@(none)$ [ 15.851142] ixgbe 0001:41:00.0 eth0: detected SFP+: 5
[ 15.990419] ixgbe 0001:41:00.0 eth0: NIC Link is Up 10 Gbps, Flow Control: RX/TX

root@(none)$ ping 192.168.20.188
PING 192.168.20.4 (192.168.20.4): 56 data bytes
64 bytes from 192.168.20.4: seq=14 ttl=128 time=1.465 ms
64 bytes from 192.168.20.4: seq=15 ttl=128 time=0.616 ms
64 bytes from 192.168.20.4: seq=16 ttl=128 time=0.391 ms
64 bytes from 192.168.20.4: seq=17 ttl=128 time=0.698 ms
64 bytes from 192.168.20.4: seq=18 ttl=128 time=0.676 ms
64 bytes from 192.168.20.4: seq=19 ttl=128 time=0.524 ms
64 bytes from 192.168.20.4: seq=20 ttl=128 time=0.301 ms
64 bytes from 192.168.20.4: seq=21 ttl=128 time=0.248 ms
64 bytes from 192.168.20.4: seq=22 ttl=128 time=0.403 ms
Sinan Kaya
2016-05-23 15:40:02 UTC
Permalink
From the functionality point of view this series may be split into the
1. New ECAM API and update for users of the pci-host-common API
2. Necessary fixes as the preparation for using driver on ARM64.
3. Use new MCFG interface and implement generic ACPI based PCI host controller driver.
4. Enable above driver on ARM64
This has been tested on Cavium ThunderX server. Any help in reviewing and
testing is very appreciated.
Tested on Qualcomm QDF2XXX server using Mellanox CX3 and Intel e1000e adapters.

Tested-by: Sinan Kaya <***@codeaurora.org>
--
Sinan Kaya
Qualcomm Technologies, Inc. on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
Continue reading on narkive:
Loading...