Trouble with handling large arrays in PowerShell 7.3 causing OutOfMemoryException
I'm experiencing an `OutOfMemoryException` when trying to process a large array in PowerShell 7.3. The array contains around 1 million objects, and I need to filter these based on a specific property. I started with the following code: ```powershell $data = 1..1000000 | ForEach-Object { [PSCustomObject]@{ Id = $_; Name = "Item$_" } } $filteredData = $data | Where-Object { $_.Id -gt 500000 } ``` However, when I run this, I get the following behavior: ``` OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. ``` To troubleshoot, I attempted to use `Select-Object` to only retrieve the properties I need before filtering: ```powershell $filteredData = $data | Select-Object Id | Where-Object { $_.Id -gt 500000 } ``` But this didn't help either. I also tried using `ForEach-Object` to process the items one by one, hoping to reduce memory consumption: ```powershell $data | ForEach-Object { if ($_.Id -gt 500000) { $_ } } ``` Yet, I still encounter the same memory scenario. I am running this script on a machine with 8GB of RAM, and I noticed that memory usage spikes significantly during execution. Are there any best practices or alternative approaches I can use to handle large datasets in PowerShell without hitting memory limits? Any advice would be greatly appreciated!