From: "Zhijian Li (Fujitsu)" To: "linux-mm@kvack.org" , "linux-cxl@vger.kernel.org" CC: "dan.j.williams@intel.com" , "Yasunori Gotou (Fujitsu)" , "david@redhat.com >> David Hildenbrand" , Oscar Salvador , "akpm@linux-foundation.org" Subject: [BUG ?] Offline Memory gets stuck in offline_pages() Date: Mon, 1 Jul 2024 01:25:30 +0000 Message-ID: <6a07125f-e720-404c-b2f9-e55f3f166e85@fujitsu.com> Accept-Language: en-US, zh-CN Content-Language: en-US Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Xref: photonic.trudheim.com org.kernel.vger.linux-cxl:29199 org.kvack.linux-mm:201950 Newsgroups: org.kernel.vger.linux-cxl,org.kvack.linux-mm Path: photonic.trudheim.com!nntp.lore.kernel.org!not-for-mail SGkgYWxsDQoNCg0KT3ZlcnZpZXc6DQpEdXJpbmcgdGVzdGluZyB0aGUgQ1hMIG1lbW9yeSBob3Ry ZW1vdmUsIHdlIG5vdGljZWQgdGhhdCBgZGF4Y3RsIG9mZmxpbmUtbWVtb3J5IGRheDAuMGANCndv dWxkIGdldCBzdHVjayBmb3JldmVyIHNvbWV0aW1lcy4gZGF4Y3RsIG9mZmxpbmUtbWVtb3J5IGRh eDAuMCB3aWxsIHdyaXRlICJvZmZsaW5lIiB0bw0KL3N5cy9kZXZpY2VzL3N5c3RlbS9tZW1vcnkv bWVtb3J5Tk5OL3N0YXRlLg0KDQpXb3JrYXJvdW5kOg0KV2hlbiBpdCBoYXBwZW5zLCB3ZSBjYW4g dHlwZSBDdHJsLUMgdG8gYWJvcnQgaXQgYW5kIHRoZW4gcmV0cnkgYWdhaW4uDQpUaGVuIHRoZSBD WEwgbWVtb3J5IGlzIGFibGUgdG8gb2ZmbGluZSBzdWNjZXNzZnVsbHkuDQoNCldoZXJlIHRoZSBr ZXJuZWwgZ2V0cyBzdHVjazoNCkFmdGVyIGRpZ2dpbmcgaW50byB0aGUga2VybmVsLCB3ZSBmb3Vu ZCB0aGF0IHdoZW4gdGhlIGlzc3VlIG9jY3VycywgdGhlIGtlcm5lbA0KaXMgc3R1Y2sgaW4gdGhl IG91dGVyIGxvb3Agb2Ygb2ZmbGluZV9wYWdlcygpLiBCZWxvdyBpcyBhIHBpZWNlIG9mIHRoZQ0K aGlnaGxpZ2h0ZWQgb2ZmbGluZV9wYWdlcygpOg0KDQpgYGANCmludCBfX3JlZiBvZmZsaW5lX3Bh Z2VzKCkNCnsNCiAgIGRvIHsgLy8gb3V0ZXIgbG9vcA0KICAgICBwZm4gPSBzdGFydF9wZm47DQog ICAgIGRvIHsNCiAgICAgICByZXQgPSBzY2FuX21vdmFibGVfcGFnZXMocGZuLCBlbmRfcGZuLCAm cGZuKTsgIC8vIEl0IHJldHVybnMgLUVOT0VOVA0KICAgICAgIGlmICghcmV0KQ0KICAgICAgICAg IGRvX21pZ3JhdGVfcmFuZ2UocGZuLCBlbmRfcGZuKTsgICAgICAgICAgICAgLy8gTm90IHJlYWNo IGhlcmUNCiAgICAgfSB3aGlsZSAoIXJldCk7DQogICAgIHJldCA9IHRlc3RfcGFnZXNfaXNvbGF0 ZWQoc3RhcnRfcGZuLCBlbmRfcGZuLCBNRU1PUllfT0ZGTElORSk7DQogICAgIH0gd2hpbGUgKHJl dCk7ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAvLyByZXQgaXMgLUVCVVNZDQp9 DQpgYGANCg0KSW4gdGhpcyBjYXNlLCB3ZSBkdW1wZWQgdGhlIGZpcnN0IHBhZ2UgdGhhdCBjYW5u b3QgYmUgaXNvbGF0ZWQgKHNlZSBkdW1wX3BhZ2UgYmVsb3cpLCBpdCdzDQpjb250ZW50IGRvZXMg bm90IGNoYW5nZSBpbiBlYWNoIGl0ZXJhdGlvbi46DQpgYGANCkp1biAyOCAxNToyOToyNiBsaW51 eCBrZXJuZWw6IHBhZ2U6IHJlZmNvdW50OjAgbWFwY291bnQ6MCBtYXBwaW5nOjAwMDAwMDAwMDAw MDAwMDAgaW5kZXg6MHgwIHBmbjoweDc5ODBkZA0KSnVuIDI4IDE1OjI5OjI2IGxpbnV4IGtlcm5l bDogZmxhZ3M6IDB4OWZmZmZmYzAwMDAwMDAobm9kZT0yfHpvbmU9M3xsYXN0Y3B1cGlkPTB4MWZm ZmZmKQ0KSnVuIDI4IDE1OjI5OjI2IGxpbnV4IGtlcm5lbDogcmF3OiAwMDlmZmZmZmMwMDAwMDAw IGZmZmZkZmJkOWU2MDM3ODggZmZmZmQ0ZjBmZmQ5N2VmMCAwMDAwMDAwMDAwMDAwMDAwDQpKdW4g MjggMTU6Mjk6MjYgbGludXgga2VybmVsOiByYXc6IDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAw MDAwMDAwMCAwMDAwMDAwMGZmZmZmZmZmIDAwMDAwMDAwMDAwMDAwMDANCkp1biAyOCAxNToyOToy NiBsaW51eCBrZXJuZWw6IHBhZ2UgZHVtcGVkIGJlY2F1c2U6IHRyb3VibGUgcGFnZS4uLg0KYGBg DQoNCkV2ZXJ5IHRpbWUgdGhlIGlzc3VlIG9jY3VycywgdGhlIGNvbnRlbnQgb2YgdGhlIHBhZ2Ug c3RydWN0dXJlIGlzIHNpbWlsYXIuDQoNClF1ZXN0aW9uczoNClExLiBJcyB0aGlzIGJlaGF2aW9y IGV4cGVjdGVkPyBBdCBsZWFzdCBmb3IgYW4gT1MgYWRtaW5pc3RyYXRvciwgaXQgc2hvdWxkIHJl dHVybg0KICAgICBwcm9tcHRseSAoc3VjY2VzcyBvciBmYWlsdXJlKSBpbnN0ZWFkIG9mIGhhbmdp bmcgaW5kZWZpbml0ZWx5Lg0KUTIuIFJlZ2FyZGluZyB0aGUgb2ZmbGluZV9wYWdlcygpIGZ1bmN0 aW9uLCBlbmNvdW50ZXJpbmcgc3VjaCBhIHBhZ2UgaW5kZWVkIGNhdXNlcw0KICAgICBhbiBlbmRs ZXNzIGxvb3AuIFNob3VsZG4ndCBhbm90aGVyIHBhcnQgb2YgdGhlIGtlcm5lbCB0aW1lbHkgY2hh bmdlZCB0aGUgc3RhdGUNCiAgICAgb2YgdGhpcyBwYWdlPw0KDQogICAgIFdoZW4gSSB1c2UgdGhl IHdvcmthcm91bmQgbWVudGlvbmVkIGFib3ZlIChDdHJsLUMgYW5kIHRyeSBvZmZsaW5lIGFnYWlu KSwgSSBmaW5kDQogICAgIHRoYXQgdGhlIHBhZ2Ugc3RhdGUgY2hhbmdlcyAoc2VlIGR1bXBfcGFn ZSBiZWxvdyk6DQpgYGANCkp1biAyOCAxNTozMzoxMiBsaW51eCBrZXJuZWw6IHBhZ2U6IHJlZmNv dW50OjAgbWFwY291bnQ6MCBtYXBwaW5nOjAwMDAwMDAwMDAwMDAwMDAgaW5kZXg6MHgwIHBmbjow eDc5ODBkZA0KSnVuIDI4IDE1OjMzOjEyIGxpbnV4IGtlcm5lbDogZmxhZ3M6IDB4OWZmZmZmYzAw MDAwMDAobm9kZT0yfHpvbmU9M3xsYXN0Y3B1cGlkPTB4MWZmZmZmKQ0KSnVuIDI4IDE1OjMzOjEy IGxpbnV4IGtlcm5lbDogcmF3OiAwMDlmZmZmZmMwMDAwMDAwIGRlYWQwMDAwMDAwMDAxMDAgZGVh ZDAwMDAwMDAwMDEyMiAwMDAwMDAwMDAwMDAwMDAwDQpKdW4gMjggMTU6MzM6MTIgbGludXgga2Vy bmVsOiByYXc6IDAwMDAwMDAwMDAwMDAwMDAgMDAwMDAwMDAwMDAwMDAwMCAwMDAwMDAwMGZmZmZm ZmZmIDAwMDAwMDAwMDAwMDAwMDANCkp1biAyOCAxNTozMzoxMiBsaW51eCBrZXJuZWw6IHBhZ2Ug ZHVtcGVkIGJlY2F1c2U6IHByZXZpb3VzIHRyb3VibGUgcGFnZQ0KYGBgDQoNCldoYXQgb3VyIHRl c3QgZG9lczoNCldlIGhhdmUgYSBDWEwgbWVtb3J5IGRldmljZSwgd2hpY2ggaXMgY29uZmlndXJl ZCBhcyBrbWVtIGFuZCBvbmxpbmUgaW50byB0aGUgTU9WQUJMRQ0Kem9uZSBhcyBOVU1BIG5vZGUy LiBXZSBydW4gdHdvIHByb2Nlc3NlcywgY29uc3VtZS1tZW1vcnkgYW5kIG9mZmxpbmUtbWVtb3J5 LCBpbiBwYXJhbGxlbCwNCnNlZSB0aGUgcHNldWRvIGNvZGUgYmVsb3c6DQoNCmBgYA0KbWFpbigp DQp7DQogICAgIGlmIChmb3JrKCkgPT0gMCkNCiAgICAgICAgIG51bWFjdGwgLW0gMiAuL2NvbnN1 bWUtbWVtb3J5DQogICAgIGVsc2Ugew0KICAgICAgICAgZGF4Y3RsIG9mZmxpbmUtbWVtb3J5IGRh eDAuMA0KICAgICAgICAgd2FpdCgpDQogICAgIH0NCn0NCmBgYA0KDQpBdHRhY2hlZCBpcyB0aGUg cHJvY2VzcyBpbmZvcm1hdGlvbiAod2hlbiBpdCBnZXRzIHN0dWNrKToNCmBgYA0Kcm9vdCAyNTcx NiAwLjAgMC4wIDI0NjAgMTQwOCBwdHMvMCBTKyAxNToyOCAwOjAwIC4vbWFpbg0Kcm9vdCAyNTcx OSAwLjAgMC4wIDAgMCBwdHMvMCBaKyAxNToyOCAwOjAwIFtjb25zdW1lLW1lbW9yeV0gPGRlZnVu Y3Q+DQpyb290IDI1NzIwIDk4LjYgMC4wIDk0NzYgMzc0MCBwdHMvMCBSKyAxNToyOCAwOjI2IGRh eGN0bCBvZmZsaW5lLW1lbW9yeSAvZGV2L2RheDAuMA0KYGBgDQoNCkZlZWwgZnJlZSB0byBsZXQg bWUga25vdyBpZiB5b3UgbmVlZCBtb3JlIGRldGFpbHMuDQpUaGFuayB5b3UgZm9yIHlvdXIgYXR0 ZW50aW9uIHRvIHRoaXMgaXNzdWUuIExvb2tpbmcgZm9yd2FyZCB0byB5b3VyIGluc2lnaHRzLg0K DQpUaGFua3MNClpoaWppYW4= . From: Robert Richter To: Alison Schofield , Vishal Verma , Ira Weiny , Dan Williams , Jonathan Cameron , Dave Jiang , Davidlohr Bueso CC: , , "Robert Richter" Subject: [PATCH v2 0/5] Address translation for HDM decoding Date: Mon, 1 Jul 2024 19:47:48 +0200 Message-ID: <20240701174754.967954-1-rrichter@amd.com> X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain Xref: photonic.trudheim.com org.kernel.vger.linux-cxl:29210 org.kernel.vger.linux-kernel:1264468 Newsgroups: org.kernel.vger.linux-cxl,org.kernel.vger.linux-kernel Path: photonic.trudheim.com!nntp.lore.kernel.org!not-for-mail Default expectation of Linux is that HPA == SPA, which means that hardware addresses in the decoders are the same as the kernel sees them. However, there are platforms where this is not the case and an address translation between decoder's (HPA) and the system's physical addresses (SPA) is needed. This series implements address translation for HDM decoding. The implementation follows the rule that the representation of hardware address ranges in the kernel are all SPA. If decoder registers (HDM decoder cap or register range) are not SPA, a base offset must be applied. Translation happens when accessing the registers back and forth. After a read access an address will be converted to SPA and before a write access the programmed address is translated from an SPA. The decoder register access can be easily encapsulated by address translation and thus there are only a few places where translation is needed and the code must be changed. This is implemented in patch #2, patch #1 is a prerequisite. Address translation is restricted to platforms that need it. As such a platform check is needed and a flag is introduced for this (patch #3). For address translation the base offset must be determined for the memory domain. Depending on the platform there are various options for this. The address range in the CEDT's CFWMS entry of the CXL host bridge can be used to determine the decoder's base address (patch #4). This is enabled for AMD Zen4 platforms (patch #5). Changelog: v2: * Fixed build error for other archs [kbot] Robert Richter (5): cxl/hdm: Moving HDM specific code to core/hdm.c. cxl/hdm: Implement address translation for HDM decoding cxl/acpi: Add platform flag for HPA address translation cxl/hdm: Setup HPA base for address translation using the HPA window in CFMWS cxl/acpi: Enable address translation for Zen4 platforms drivers/cxl/acpi.c | 16 +++ drivers/cxl/core/core.h | 2 + drivers/cxl/core/hdm.c | 245 ++++++++++++++++++++++++++++++++++++++-- drivers/cxl/core/pci.c | 119 +------------------ drivers/cxl/cxl.h | 2 + drivers/cxl/cxlmem.h | 4 + drivers/cxl/cxlpci.h | 3 - 7 files changed, 262 insertions(+), 129 deletions(-) base-commit: 1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0 -- 2.39.2 . Message-ID: Date: Mon, 1 Jul 2024 10:53:44 -0700 X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Language: en-US To: Linus Torvalds Cc: "linux-cxl@vger.kernel.org" , Dan Williams , Davidlohr Bueso , Jonathan Cameron , Ira Weiny , Vishal Verma , Alison Schofield From: Dave Jiang Subject: [GIT PULL] Compute Express Link (CXL) Fixes for 6.10-rc7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Xref: photonic.trudheim.com org.kernel.vger.linux-cxl:29217 Newsgroups: org.kernel.vger.linux-cxl Path: photonic.trudheim.com!nntp.lore.kernel.org!not-for-mail Hi Linus, please pull from git://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git tags/cxl-fixes-6.10-rc7 ...to receive several fixes for the CXL subsystem. A fix to deal with NULL cxl_nvd pointer during persistent memory auto assembly. A fix to address NULL pointer dereference in CXL region lookup. A fix to add proper checking for CXL region interleave capability during assembly. A kdoc fix for CXL documentation. This has build success notification from kbuild-robot. It has been in the -next for several days with no reported issues. --- The following changes since commit 6ba59ff4227927d3a8530fc2973b80e94b54d58f: Linux 6.10-rc4 (2024-06-16 13:40:16 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git tags/cxl-fixes-6.10-rc7 for you to fetch changes up to a0f39d51dbf72c28283bd201b97559ed82bc0fe5: cxl: documentation: add missing files to cxl driver-api (2024-06-25 14:45:27 -0700) ---------------------------------------------------------------- cxl fixes for v6.10-rc7 - Fix no cxl_nvd during pmem region auto-assemble - Avoid NULLL pointer dereference in region lookup - Add missing checks to interleave capability - Add cxl kdoc fix to address document compilation error ---------------------------------------------------------------- Alison Schofield (1): cxl/region: Avoid null pointer dereference in region lookup Li Ming (1): cxl/mem: Fix no cxl_nvd during pmem region auto-assembling Yao Xingtao (2): cxl/region: check interleave capability cxl: documentation: add missing files to cxl driver-api Documentation/driver-api/cxl/memory-devices.rst | 15 ++++ drivers/cxl/core/hdm.c | 13 +++ drivers/cxl/core/pmem.c | 16 ++-- drivers/cxl/core/region.c | 103 ++++++++++++++++++++++-- drivers/cxl/cxl.h | 6 +- drivers/cxl/cxlmem.h | 21 +++-- drivers/cxl/mem.c | 17 ++-- tools/testing/cxl/test/cxl.c | 4 + 8 files changed, 170 insertions(+), 25 deletions(-) . From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Subject: [PATCH v6 0/2] cxl: Region bandwidth calculation for targets with shared upstream link Date: Mon, 1 Jul 2024 14:49:13 -0700 Message-ID: <20240701215020.3813275-1-dave.jiang@intel.com> X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Xref: photonic.trudheim.com org.kernel.vger.linux-cxl:29221 org.kernel.vger.linux-pci:144762 Newsgroups: org.kernel.vger.linux-cxl,org.kernel.vger.linux-pci Path: photonic.trudheim.com!nntp.lore.kernel.org!not-for-mail v6: - Update kdoc in comments. (Ira) - Various rearranging and cleanup of variable declarations. (Jonathan) This series provides recalculation of the CXL region bandwidth when the targets have shared upstream link by walking the toplogy from bottom up and clamp the bandwdith as the code traverses up the tree. An example topology: An example topology from Jonathan: CFMWS 0 | _________|_________ | | ACPI0017-0 ACPI0017-1 GP0/HB0/ACPI0016-0 GP1/HB1/ACPI0016-1 | | | | RP0 RP1 RP2 RP3 | | | | SW 0 SW 1 SW 2 SW 3 | | | | | | | | EP0 EP1 EP2 EP3 EP4 EP5 EP6 EP7 Computation for the example topology: Min (GP0 to CPU BW, Min(SW 0 Upstream Link to RP0 BW, Min(SW0SSLBIS for SW0DSP0 (EP0), EP0 DSLBIS, EP0 Upstream Link) + Min(SW0SSLBIS for SW0DSP1 (EP1), EP1 DSLBIS, EP1 Upstream link)) + Min(SW 1 Upstream Link to RP1 BW, Min(SW1SSLBIS for SW1DSP0 (EP2), EP2 DSLBIS, EP2 Upstream Link) + Min(SW1SSLBIS for SW1DSP1 (EP3), EP3 DSLBIS, EP3 Upstream link))) + Min (GP1 to CPU BW, Min(SW 2 Upstream Link to RP2 BW, Min(SW2SSLBIS for SW2DSP0 (EP4), EP4 DSLBIS, EP4 Upstream Link) + Min(SW2SSLBIS for SW2DSP1 (EP5), EP5 DSLBIS, EP5 Upstream link)) + Min(SW 3 Upstream Link to RP3 BW, Min(SW3SSLBIS for SW3DSP0 (EP6), EP6 DSLBIS, EP6 Upstream Link) + Min(SW3SSLBIS for SW3DSP1 (EP7), EP7 DSLBIS, EP7 Upstream link)))) .