c# - 数据集级联和内存消耗

标签 c# .net memory memory-management dataset

我有一个应用程序，它使用 DataSet.WriteXML 导出数据并使用 DataSet.ReadXML 导入数据。在导入过程中，我需要更改某些主键作为应用程序逻辑的一部分。

当记录超过 500K 时，它会成功写入 XML 并从 XML 中读取。一旦我更改了主键，它会等待一段时间并引发 OutOfMemory 异常。我相信的原因是，它必须进行大量级联更新。我在主键更改期间尝试了 BeginEdit 和 EndEdit，但在这种情况下仍然在 EndEdit 中失败。

据我了解，DataSets 还将一些以前的数据保存在内存中。有什么方法可以优化 DataSet 更新操作以消耗最少的内存？

最佳答案

如果您需要更多控制权，则需要删除数据集为您提供的一些功能。减少由级联引起的内存的一种方法是简单的不要级联。使用表模式手动更新表 ID。

这个想法是您可以控制更新哪些行、随时接受更改、强制 GC 中间更新或您可能想要控制的任何其他内容。

我创建了一个简单的测试场景来说明我的意思:

enter image description here

架构:

<?xml version="1.0"?>
<xs:schema id="NewDataSet" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
  <xs:element name="NewDataSet" msdata:IsDataSet="true" msdata:UseCurrentLocale="true">
    <xs:complexType>
      <xs:choice minOccurs="0" maxOccurs="unbounded">
        <xs:element name="Planet">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="ID" type="xs:int" />
              <xs:element name="Name" type="xs:string" minOccurs="0" />
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element name="Continent">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="ID" type="xs:int" />
              <xs:element name="PlanetID" type="xs:int" />
              <xs:element name="Name" type="xs:string" minOccurs="0" />
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element name="Country">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="ID" type="xs:int" />
              <xs:element name="ContinentID" type="xs:int" />
              <xs:element name="Name" type="xs:string" minOccurs="0" />
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element name="County">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="ID" type="xs:int" />
              <xs:element name="CountryID" type="xs:int" />
              <xs:element name="Name" type="xs:string" minOccurs="0" />
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element name="City">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="ID" type="xs:int" />
              <xs:element name="CountyID" type="xs:int" />
              <xs:element name="Name" type="xs:string" minOccurs="0" />
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element name="Street">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="ID" type="xs:int" />
              <xs:element name="CityID" type="xs:int" minOccurs="0" />
              <xs:element name="Name" type="xs:string" minOccurs="0" />
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element name="People">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="ID" type="xs:int" />
              <xs:element name="StreetID" type="xs:int" />
              <xs:element name="Name" type="xs:string" minOccurs="0" />
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element name="Job">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="ID" type="xs:int" />
              <xs:element name="PeopleID" type="xs:int" />
              <xs:element name="Name" type="xs:string" minOccurs="0" />
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element name="Pets">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="ID" type="xs:int" />
              <xs:element name="PeopleID" type="xs:int" minOccurs="0" />
              <xs:element name="Name" type="xs:string" minOccurs="0" />
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:choice>
    </xs:complexType>
    <xs:unique name="Constraint1">
      <xs:selector xpath=".//Planet" />
      <xs:field xpath="ID" />
    </xs:unique>
    <xs:unique name="Continent_Constraint1" msdata:ConstraintName="Constraint1">
      <xs:selector xpath=".//Continent" />
      <xs:field xpath="ID" />
    </xs:unique>
    <xs:unique name="Country_Constraint1" msdata:ConstraintName="Constraint1">
      <xs:selector xpath=".//Country" />
      <xs:field xpath="ID" />
    </xs:unique>
    <xs:unique name="County_Constraint1" msdata:ConstraintName="Constraint1">
      <xs:selector xpath=".//County" />
      <xs:field xpath="ID" />
    </xs:unique>
    <xs:unique name="City_Constraint1" msdata:ConstraintName="Constraint1">
      <xs:selector xpath=".//City" />
      <xs:field xpath="ID" />
    </xs:unique>
    <xs:unique name="Street_Constraint1" msdata:ConstraintName="Constraint1">
      <xs:selector xpath=".//Street" />
      <xs:field xpath="ID" />
    </xs:unique>
    <xs:unique name="People_Constraint1" msdata:ConstraintName="Constraint1">
      <xs:selector xpath=".//People" />
      <xs:field xpath="ID" />
    </xs:unique>
    <xs:unique name="Job_Constraint1" msdata:ConstraintName="Constraint1">
      <xs:selector xpath=".//Job" />
      <xs:field xpath="ID" />
    </xs:unique>
    <xs:unique name="Pets_Constraint1" msdata:ConstraintName="Constraint1">
      <xs:selector xpath=".//Pets" />
      <xs:field xpath="ID" />
    </xs:unique>
    <xs:keyref name="Relation8" refer="People_Constraint1">
      <xs:selector xpath=".//Pets" />
      <xs:field xpath="PeopleID" />
    </xs:keyref>
    <xs:keyref name="Relation7" refer="People_Constraint1">
      <xs:selector xpath=".//Job" />
      <xs:field xpath="PeopleID" />
    </xs:keyref>
    <xs:keyref name="Relation6" refer="Street_Constraint1">
      <xs:selector xpath=".//People" />
      <xs:field xpath="StreetID" />
    </xs:keyref>
    <xs:keyref name="Relation5" refer="City_Constraint1">
      <xs:selector xpath=".//Street" />
      <xs:field xpath="CityID" />
    </xs:keyref>
    <xs:keyref name="Relation4" refer="County_Constraint1">
      <xs:selector xpath=".//City" />
      <xs:field xpath="CountyID" />
    </xs:keyref>
    <xs:keyref name="Relation3" refer="Country_Constraint1">
      <xs:selector xpath=".//County" />
      <xs:field xpath="CountryID" />
    </xs:keyref>
    <xs:keyref name="Relation2" refer="Continent_Constraint1">
      <xs:selector xpath=".//Country" />
      <xs:field xpath="ContinentID" />
    </xs:keyref>
    <xs:keyref name="Relation1" refer="Constraint1">
      <xs:selector xpath=".//Continent" />
      <xs:field xpath="PlanetID" />
    </xs:keyref>
  </xs:element>
</xs:schema>

还有一些生成测试用例的代码

    private void CreateRows(Int32 MaxBaseRows, Int32 MaxChildRows)
    {
        dataSet1.Clear();
        Int32 RowCount = 0;
        Random R = new Random();
        foreach (DataTable DT in dataSet1.Tables)
        {
            Int32 NewCount = R.Next(1, MaxBaseRows);
            foreach (var FK in DT.Constraints.OfType<ForeignKeyConstraint>())
            {
                NewCount = NewCount * R.Next(1, MaxChildRows);
            }
            for (int i = 0; i < NewCount; i++)
            {
                DataRow DR = DT.NewRow();
                foreach (DataColumn DC in DT.Columns)
                {
                    if (DC.ColumnName == "ID")
                    {
                        DR[DC] = DT.Rows.Count;
                    }
                    else if (DC.DataType == typeof(Int32))
                    {
                        Boolean ValueSet = false;
                        foreach (var FK in DT.Constraints.OfType<ForeignKeyConstraint>())
                        {
                            if (FK.Columns.Contains(DC))
                            {
                                DR[DC] = R.Next(0, FK.RelatedTable.Rows.Count);
                                ValueSet = true;
                            }
                        }
                        if (!ValueSet)
                        {
                            DR[DC] = R.Next(0, 10000);
                        }
                    }
                    else if (DC.DataType == typeof(String))
                    {
                        DR[DC] = String.Format("{0}{1}", DT.TableName, DT.Rows.Count);
                    }
                }
                DT.Rows.Add(DR);
                RowCount++;
            }
        }
        label19.Text = RowCount.ToString();
        dataSet1.AcceptChanges();
    }


    private void UpdateUsingCascade()
    {
        EnableRelations();
        GC.Collect();
        long Mem = System.GC.GetTotalMemory(false);
        if (dataSet1.Tables["Planet"].Rows.Count > 0)
        {
            dataSet1.Tables["Planet"].Rows[0]["ID"] = new Random().Next(BaseRowCount, BaseRowCount + 10);
        }
        Mem = System.GC.GetTotalMemory(false) - Mem;
        DataSet ds = dataSet1.GetChanges();
        Int32 Changes = ds.Tables.OfType<DataTable>().Sum(DT => DT.Rows.Count);
        label19.Text = Changes.ToString();
        label21.Text = Mem.ToString();
        dataSet1.AcceptChanges();
    }

    private void UpdateManually()
    {
        DisableRelations();
        GC.Collect();
        long Mem = System.GC.GetTotalMemory(false);

        DataTable DT = dataSet1.Tables["Planet"];
        Int32 ChangeCount = 0;
        if (DT.Rows.Count > 0)
        {
            DataColumn DC = DT.Columns["ID"];
            Int32 oldValue = Convert.ToInt32(DT.Rows[0][DC]);
            DT.Rows[0][DC] = new Random().Next(BaseRowCount + 20,BaseRowCount + 30);
            Int32 newValue = Convert.ToInt32(DT.Rows[0][DC]);
            foreach (DataRelation Relation in DT.ChildRelations)
            {
                if (Relation.ParentColumns.Contains(DC))
                {
                    foreach (DataColumn CC in Relation.ChildColumns)
                    {
                        foreach (DataRow DR in Relation.ChildTable.Rows)
                        {
                            if (Convert.ToInt32(DR[CC]) == oldValue)
                            {
                                DR[CC] = newValue;
                                ChangeCount++;
                                dataSet1.AcceptChanges();
                                GC.Collect();
                            }
                        }
                    }
                }
            }
        }
        Mem = System.GC.GetTotalMemory(false) - Mem;
        label20.Text = ChangeCount.ToString();
        label22.Text = Mem.ToString();
        dataSet1.AcceptChanges();
    }

    private void EnableRelations()
    {
        dataSet1.EnforceConstraints = true;
        foreach (DataRelation Relation in dataSet1.Relations)
        {
            Relation.ChildKeyConstraint.UpdateRule = Rule.Cascade;
        }
    }

    private void DisableRelations()
    {
        dataSet1.EnforceConstraints = false;
        foreach (DataRelation Relation in dataSet1.Relations)
        {
            Relation.ChildKeyConstraint.UpdateRule = Rule.None;
        }
    }

关于c# - 数据集级联和内存消耗，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/15058436/

上一篇：android - 旋转图像时如何避免 OutOfMemory ex？

下一篇：php memory_get_usage(true) vs top %MEM

相关文章：

c# - EF4 代码优先 : how to add a relationship without adding a navigation property

c# - 如何从该类中传递该类的实例

c# - ASP.NET 响应 URL 重定向错误；从静态方法

c# - Entity Framework 中的多对多排序

c++ - Malloc 似乎比 new[] 使用更少的内存

c++ - 在自定义硬件上映射内存

c# - 如何有效地从存储过程中检索数千行

c# - 将自定义日期格式(字符串)转换为日期时间

c# - 如何从 Combobox 中设置选定的值？

java - 我的java程序可以使用磁盘作为大对象的堆吗？

©2024 IT工具网  联系我们