rss.xml

<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <atom:link href="https://jiangxj.top/vuepress/vuepress/rss.xml" rel="self" type="application/rss+xml"/>
    <title>Docs Library by #XJ</title>
    <link>https://jiangxj.top/vuepress/vuepress/</link>
    <description>Vue-driven documents generator based on vuepress-theme-hope</description>
    <language>zh-CN</language>
    <pubDate>Sun, 07 Jan 2024 05:47:57 GMT</pubDate>
    <lastBuildDate>Sun, 07 Jan 2024 05:47:57 GMT</lastBuildDate>
    <generator>@mr-hope/vuepress-plugin-feed</generator>
    <docs>https://validator.w3.org/feed/docs/rss2.html</docs>
    <category>Guide</category>
    <category>guide</category>
    <category>使用指南</category>
    <category>深度强化学习</category>
    <category>MachineLearning</category>
    <category>自然语言处理</category>
    <category>ComputerScience</category>
    <category>Maths</category>
    <category>NuclearScience</category>
    <category>PapersReading</category>
    <item>
      <title>Component disabled</title>
      <link>https://jiangxj.top/vuepress/vuepress/en/guide/disable/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/en/guide/disable/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Component disabled</source>
      <description>&amp;lt;p&amp;gt;You can disable some functions on the page by setting the Frontmatter of the page.&amp;lt;/p&amp;gt;
</description>
      <category>Guide</category>
      <content:encoded><![CDATA[<p>You can disable some functions on the page by setting the Frontmatter of the page.</p>

<p>These should be disabled on this page:</p>
<ul>
<li>Navbar</li>
<li>Sidebar</li>
<li>Breadcrumb</li>
<li>Page information</li>
<li>Contributors</li>
<li>Edit link</li>
<li>Update time</li>
<li>Prev/Next link</li>
<li>Comment</li>
<li>Footer</li>
<li>Back to top button</li>
</ul>
]]></content:encoded>
    </item>
    <item>
      <title>Markdown Enhance</title>
      <link>https://jiangxj.top/vuepress/vuepress/en/guide/markdown/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/en/guide/markdown/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Markdown Enhance</source>
      <description>&amp;lt;p&amp;gt;&amp;lt;code&amp;gt;vuepress-theme-hope&amp;lt;/code&amp;gt; enables more syntax in Markdown via the built-in &amp;lt;a href=&amp;quot;https://vuepress-theme-hope.github.io/v1/md-enhance/&amp;quot; target=&amp;quot;_blank&amp;quot; rel=&amp;quot;noopener noreferrer&amp;quot;&amp;gt;md-enhance&amp;lt;/a&amp;gt; plugin.&amp;lt;/p&amp;gt;
</description>
      <category>Guide</category>
      <content:encoded><![CDATA[<p><code>vuepress-theme-hope</code> enables more syntax in Markdown via the built-in <a href="https://vuepress-theme-hope.github.io/v1/md-enhance/" target="_blank" rel="noopener noreferrer">md-enhance</a> plugin.</p>

<h2 id="enable-all"> Enable all</h2>
<p>You can set <code>themeconfig.mdEnhance.enableAll</code> to enable all features of the <a href="https://vuepress-theme-hope.github.io/v1/md-enhance/" target="_blank" rel="noopener noreferrer">md-enhance</a> plugin.</p>
<div><div><br><br><div>&nbsp;</div><div>&nbsp;</div><div>&nbsp;</div><br><br><br></div><pre><code>module<span>.</span>exports <span>=</span> <span>{</span>
  <span>themeConfig</span><span>:</span> <span>{</span>
    <span>mdEnhance</span><span>:</span> <span>{</span>
      <span>enableAll</span><span>:</span> <span>true</span><span>,</span>
    <span>}</span><span>,</span>
  <span>}</span><span>,</span>
<span>}</span><span>;</span>
</code></pre><div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br></div></div><h2 id="new-feature"> New Feature</h2>
<h3 id="superscript-and-subscript"> Superscript and Subscript</h3>
<p>19<sup>th</sup> H<sub>2</sub>O</p>
<details><summary>Code</summary>
<div><pre><code>19^th^ H<span><span>~</span><span>2</span><span>~</span></span>O
</code></pre>
<div><span>1</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/v1/guide/markdown/sup-sub.html" target="_blank" rel="noopener noreferrer">View Detail</a></li>
</ul>
<h3 id="align"> Align</h3>
<p>::: center</p>
<p>I am center</p>
<p>:::</p>
<p>::: right</p>
<p>I am right align</p>
<p>:::</p>
<details><summary>Code</summary>
<div><pre><code>::: center

I am center

:::

::: right

I am right align

:::
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/v1/guide/markdown/align.html" target="_blank" rel="noopener noreferrer">View Detail</a></li>
</ul>
<h3 id="footnote"> Footnote</h3>
<p>This text has footnote<sup></sup>.</p>
<details><summary>Code</summary>
<div><pre><code>This text has footnote[^first].

<span><span>[</span><span>^first</span><span>]</span><span>:</span> This</span> is footnote content
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/v1/guide/markdown/footnote.html" target="_blank" rel="noopener noreferrer">View Detail</a></li>
</ul>
<h3 id="mark"> Mark</h3>
<p>You can mark ==important words== .</p>
<details><summary>Code</summary>
<div><pre><code>You can mark ==important words== .
</code></pre>
<div><span>1</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/v1/guide/markdown/mark.html" target="_blank" rel="noopener noreferrer">View Detail</a></li>
</ul>
<h3 id="tasklist"> Tasklist</h3>
<ul>
<li>[x] Plan A</li>
<li>[ ] Plan B</li>
</ul>
<details><summary>Code</summary>
<div><pre><code><span>-</span> [x] Plan A
<span>-</span> [ ] Plan B
</code></pre>
<div><span>1</span><br><span>2</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/v1/guide/markdown/tasklist.html" target="_blank" rel="noopener noreferrer">View Detail</a></li>
</ul>
<h3 id="flowchart"> Flowchart</h3>
<i>Not supported content</i><details><summary>Code</summary>
<div><pre><code><span><span>```</span><span>flow</span>
<span>cond=>condition: Process?
process=>operation: Process
e=>end: End

cond(yes)->process->e
cond(no)->e</span>
<span>```</span></span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/v1/guide/markdown/flowchart.html" target="_blank" rel="noopener noreferrer">View Detail</a></li>
</ul>
<h2 id="mermaid"> Mermaid</h2>
<Mermaid id="mermaid-64a57026" data-code="graph%20TD%3B%0A%20%20%20%20A--%3EB%3B%0A%20%20%20%20A--%3EC%3B%0A%20%20%20%20B--%3ED%3B%0A%20%20%20%20C--%3ED%3B%0A"></Mermaid><details><summary>Code</summary>
<div><pre><code><span><span>```</span><span>mermaid</span>
<span>graph TD;
    A-->B;
    A-->C;
    B-->D;
    C-->D;</span>
<span>```</span></span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/v1/guide/markdown/mermaid.html" target="_blank" rel="noopener noreferrer">View Detail</a></li>
</ul>
<h3 id="tex"> Tex</h3>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:2.4em;vertical-align:-0.95em;"></span><span><span></span><span><span><span><span style="height:1.3714em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span style="margin-right:0.05556em;">∂</span><span><span style="margin-right:0.03588em;">ω</span><span><span><span><span style="height:0.5904em;"><span style="top:-2.989em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.02778em;">r</span></span></span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.05556em;">∂</span><span><span><span><span style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.02778em;">r</span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;"><span>(</span></span><span><span></span><span><span><span><span style="height:1.3414em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span style="margin-right:0.03588em;">ω</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span style="margin-right:0.03588em;">ω</span></span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="top:0em;"><span>)</span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:3.0277em;vertical-align:-1.2777em;"></span><span><span style="top:0em;"><span>(</span></span><span><span></span><span><span><span><span style="height:1.3414em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span style="margin-right:0.03588em;">ω</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span style="margin-right:0.03588em;">ω</span></span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="top:0em;"><span>)</span></span></span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;"><span>{</span></span><span>(</span><span>lo<span style="margin-right:0.01389em;">g</span></span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span><span>)</span><span><span><span><span style="height:0.7144em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.02778em;">r</span></span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span style="margin-right:0.02778em;">r</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span></span><span><span><span><span style="height:1.5017em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.03588em;">ω</span><span><span><span><span style="height:0.7507em;"><span style="top:-2.989em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>(</span><span>−</span><span>1</span><span><span>)</span><span><span><span><span style="height:0.8247em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span></span></span></span></span><span style="margin-right:0.02778em;">r</span><span style="margin-right:0.1667em;"></span><span>⋯</span><span style="margin-right:0.1667em;"></span><span>(</span><span style="margin-right:0.02778em;">r</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span>i</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span>1</span><span>)</span><span>(</span><span>lo<span style="margin-right:0.01389em;">g</span></span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span><span>)</span><span><span><span><span style="height:0.8247em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span style="margin-right:0.02778em;">r</span><span>−</span><span>i</span></span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="top:0em;"><span>}</span></span></span></span></span></span></span></p>
<details><summary>Code</summary>
<div><pre><code>$$
\frac {\partial^r} {\partial \omega^r} \left(\frac {y^{\omega}} {\omega}\right)
= \left(\frac {y^{\omega}} {\omega}\right) \left\{(\log y)^r + \sum_{i=1}^r \frac {(-1)^i r \cdots (r-i+1) (\log y)^{r-i}} {\omega^i} \right\}
$$
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/v1/guide/markdown/tex.html" target="_blank" rel="noopener noreferrer">View Detail</a></li>
</ul>
<h3 id="demo"> Demo</h3>

          <div
            id="code-demo-688e9568"
           
  
 data-title="A%20normal%20demo"

            data-code="%7B%22html%22%3A%22%3Ch1%3EMr.Hope%3C%2Fh1%3E%5Cn%3Cp%3EIs%20%3Cspan%20id%3D%5C%22very%5C%22%3Every%3C%2Fspan%3E%20handsome%3C%2Fp%3E%5Cn%22%2C%22js%22%3A%22document.querySelector(%5C%22%23very%5C%22).addEventListener(%5C%22click%5C%22%2C%20()%20%3D%3E%20%7B%5Cn%20%20alert(%5C%22Very%20handsome!%5C%22)%3B%5Cn%7D)%3B%5Cn%22%2C%22css%22%3A%22span%20%7B%5Cn%20%20color%3A%20red%3B%5Cn%7D%5Cn%22%7D"
          >
              
              <div>
                <div>
<div><pre><code><span><span><span>&lt;</span>h1</span><span>></span></span>Mr.Hope<span><span><span>&lt;/</span>h1</span><span>></span></span>
<span><span><span>&lt;</span>p</span><span>></span></span>Is <span><span><span>&lt;</span>span</span> <span>id</span><span><span>=</span><span>"</span>very<span>"</span></span><span>></span></span>very<span><span><span>&lt;/</span>span</span><span>></span></span> handsome<span><span><span>&lt;/</span>p</span><span>></span></span>
</code></pre>
<div><span>1</span><br><span>2</span><br></div></div><div><pre><code>document<span>.</span><span>querySelector</span><span>(</span><span>"#very"</span><span>)</span><span>.</span><span>addEventListener</span><span>(</span><span>"click"</span><span>,</span> <span>(</span><span>)</span> <span>=></span> <span>{</span>
  <span>alert</span><span>(</span><span>"Very handsome!"</span><span>)</span><span>;</span>
<span>}</span><span>)</span><span>;</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><div><pre><code><span>span</span> <span>{</span>
  <span>color</span><span>:</span> red<span>;</span>
<span>}</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div>
            </div>
          </div>
          
        </div>
<details><summary>Code</summary>
<div><pre><code>::: demo A normal demo

<span><span>```</span><span>html</span>
<span><span><span><span>&lt;</span>h1</span><span>></span></span>Mr.Hope<span><span><span>&lt;/</span>h1</span><span>></span></span>
<span><span><span>&lt;</span>p</span><span>></span></span>Is <span><span><span>&lt;</span>span</span> <span>id</span><span><span>=</span><span>"</span>very<span>"</span></span><span>></span></span>very<span><span><span>&lt;/</span>span</span><span>></span></span> handsome<span><span><span>&lt;/</span>p</span><span>></span></span></span>
<span>```</span></span>

<span><span>```</span><span>js</span>
<span>document<span>.</span><span>querySelector</span><span>(</span><span>"#very"</span><span>)</span><span>.</span><span>addEventListener</span><span>(</span><span>"click"</span><span>,</span> <span>(</span><span>)</span> <span>=></span> <span>{</span>
  <span>alert</span><span>(</span><span>"Very handsome!"</span><span>)</span><span>;</span>
<span>}</span><span>)</span><span>;</span></span>
<span>```</span></span>

<span><span>```</span><span>css</span>
<span><span>span</span> <span>{</span>
  <span>color</span><span>:</span> red<span>;</span>
<span>}</span></span>
<span>```</span></span>

:::
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br></div></div></details>

          <div
            id="code-demo-4ead1181"
           
  data-type="react"
 data-title="A%20react%20demo"

            data-code="%7B%22js%22%3A%22export%20default%20class%20App%20extends%20React.Component%20%7B%5Cn%20%20constructor(props)%20%7B%5Cn%20%20%20%20super(props)%3B%5Cn%20%20%20%20this.state%20%3D%20%7B%20message%3A%20%5C%22very%20handsome%5C%22%20%7D%3B%5Cn%20%20%7D%5Cn%20%20render()%20%7B%5Cn%20%20%20%20return%20(%5Cn%20%20%20%20%20%20%3Cdiv%20className%3D%5C%22box-react%5C%22%3E%5Cn%20%20%20%20%20%20%20%20Mr.Hope%20is%20%3Cspan%3E%7Bthis.state.message%7D%3C%2Fspan%3E%5Cn%20%20%20%20%20%20%3C%2Fdiv%3E%5Cn%20%20%20%20)%3B%5Cn%20%20%7D%5Cn%7D%5Cn%22%2C%22css%22%3A%22.box-react%20span%20%7B%5Cn%20%20color%3A%20red%3B%5Cn%7D%5Cn%22%7D"
          >
              
              <div>
                <div>
<div><pre><code><span>export</span> <span>default</span> <span>class</span> <span>App</span> <span>extends</span> <span>React<span>.</span>Component</span> <span>{</span>
  <span>constructor</span><span>(</span><span>props</span><span>)</span> <span>{</span>
    <span>super</span><span>(</span>props<span>)</span><span>;</span>
    <span>this</span><span>.</span>state <span>=</span> <span>{</span> <span>message</span><span>:</span> <span>"very handsome"</span> <span>}</span><span>;</span>
  <span>}</span>
  <span>render</span><span>(</span><span>)</span> <span>{</span>
    <span>return</span> <span>(</span>
      <span>&lt;</span>div className<span>=</span><span>"box-react"</span><span>></span>
        Mr<span>.</span>Hope is <span>&lt;</span>span<span>></span><span>{</span><span>this</span><span>.</span>state<span>.</span>message<span>}</span><span>&lt;</span><span>/</span>span<span>></span>
      <span>&lt;</span><span>/</span>div<span>></span>
    <span>)</span><span>;</span>
  <span>}</span>
<span>}</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br></div></div><div><pre><code><span>.box-react span</span> <span>{</span>
  <span>color</span><span>:</span> red<span>;</span>
<span>}</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div>
            </div>
          </div>
          
        </div>
<details><summary>Code</summary>
<div><pre><code>::: demo [react] A react demo

<span><span>```</span><span>js</span>
<span><span>export</span> <span>default</span> <span>class</span> <span>App</span> <span>extends</span> <span>React<span>.</span>Component</span> <span>{</span>
  <span>constructor</span><span>(</span><span>props</span><span>)</span> <span>{</span>
    <span>super</span><span>(</span>props<span>)</span><span>;</span>
    <span>this</span><span>.</span>state <span>=</span> <span>{</span> <span>message</span><span>:</span> <span>"very handsome"</span> <span>}</span><span>;</span>
  <span>}</span>
  <span>render</span><span>(</span><span>)</span> <span>{</span>
    <span>return</span> <span>(</span>
      <span>&lt;</span>div className<span>=</span><span>"box-react"</span><span>></span>
        Mr<span>.</span>Hope is <span>&lt;</span>span<span>></span><span>{</span><span>this</span><span>.</span>state<span>.</span>message<span>}</span><span>&lt;</span><span>/</span>span<span>></span>
      <span>&lt;</span><span>/</span>div<span>></span>
    <span>)</span><span>;</span>
  <span>}</span>
<span>}</span></span>
<span>```</span></span>

<span><span>```</span><span>css</span>
<span><span>.box-react span</span> <span>{</span>
  <span>color</span><span>:</span> red<span>;</span>
<span>}</span></span>
<span>```</span></span>

:::
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br><span>22</span><br><span>23</span><br><span>24</span><br><span>25</span><br></div></div></details>

          <div
            id="code-demo-4edf655c"
           
  data-type="vue"
 data-title="A%20vue%20demo"

            data-code="%7B%22vue%22%3A%22%3Ctemplate%3E%5Cn%20%20%3Cdiv%20class%3D%5C%22box%5C%22%3E%5Cn%20%20%20%20Mr.Hope%20is%20%3Cspan%3E%7B%7B%20message%20%7D%7D%3C%2Fspan%3E%5Cn%20%20%3C%2Fdiv%3E%5Cn%3C%2Ftemplate%3E%5Cn%3Cscript%3E%5Cnexport%20default%20%7B%5Cn%20%20data%3A%20()%20%3D%3E%20(%7B%20message%3A%20%5C%22very%20handsome%5C%22%20%7D)%2C%5Cn%7D%3B%5Cn%3C%2Fscript%3E%5Cn%3Cstyle%3E%5Cn.box%20span%20%7B%5Cn%20%20color%3A%20red%3B%5Cn%7D%5Cn%3C%2Fstyle%3E%5Cn%22%7D"
          >
              
              <div>
                <div>
<div><pre><code><span><span><span>&lt;</span>template</span><span>></span></span>
  <span><span><span>&lt;</span>div</span> <span>class</span><span><span>=</span><span>"</span>box<span>"</span></span><span>></span></span>
    Mr.Hope is <span><span><span>&lt;</span>span</span><span>></span></span>{{ message }}<span><span><span>&lt;/</span>span</span><span>></span></span>
  <span><span><span>&lt;/</span>div</span><span>></span></span>
<span><span><span>&lt;/</span>template</span><span>></span></span>
<span><span><span>&lt;</span>script</span><span>></span></span><span><span>
<span>export</span> <span>default</span> <span>{</span>
  <span>data</span><span>:</span> <span>(</span><span>)</span> <span>=></span> <span>(</span><span>{</span> <span>message</span><span>:</span> <span>"very handsome"</span> <span>}</span><span>)</span><span>,</span>
<span>}</span><span>;</span>
</span></span><span><span><span>&lt;/</span>script</span><span>></span></span>
<span><span><span>&lt;</span>style</span><span>></span></span><span><span>
<span>.box span</span> <span>{</span>
  <span>color</span><span>:</span> red<span>;</span>
<span>}</span>
</span></span><span><span><span>&lt;/</span>style</span><span>></span></span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br></div></div>
            </div>
          </div>
          
        </div>
<details><summary>Code</summary>
<div><pre><code>::: demo [vue] A vue demo

<span><span>```</span><span>vue</span>
<span>&lt;template>
  &lt;div>
    Mr.Hope is &lt;span>{{ message }}&lt;/span>
  &lt;/div>
&lt;/template>
&lt;script>
export default {
  data: () => ({ message: "very handsome" }),
};
&lt;/script>
&lt;style>
.box span {
  color: red;
}
&lt;/style></span>
<span>```</span></span>

:::
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br></div></div></details>

          <div
            id="code-demo-2b1f2266"
           
  
 data-title="A%20normal%20demo"

            data-code="%7B%22md%22%3A%22%23%20Title%5Cn%5Cnis%20very%20handsome.%5Cn%22%2C%22ts%22%3A%22const%20message%3A%20string%20%3D%20%5C%22Mr.Hope%5C%22%3B%5Cn%5Cndocument.querySelector(%5C%22h1%5C%22).innerHTML%20%3D%20message%3B%5Cn%22%2C%22scss%22%3A%22h1%20%7B%5Cn%20%20font-style%3A%20italic%3B%5Cn%5Cn%20%20%2B%20p%20%7B%5Cn%20%20%20%20color%3A%20red%3B%5Cn%20%20%7D%5Cn%7D%5Cn%22%7D"
          >
              
              <div>
                <div>
<div><pre><code><span><span>#</span> Title</span>

is very handsome.
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><div><pre><code><span>const</span> message<span>:</span> <span>string</span> <span>=</span> <span>"Mr.Hope"</span><span>;</span>

document<span>.</span><span>querySelector</span><span>(</span><span>"h1"</span><span>)</span><span>.</span>innerHTML <span>=</span> message<span>;</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><div><pre><code><span>h1 </span><span>{</span>
  <span>font-style</span><span>:</span> italic<span>;</span>

  <span>+ p </span><span>{</span>
    <span>color</span><span>:</span> red<span>;</span>
  <span>}</span>
<span>}</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br></div></div>
            </div>
          </div>
          
        </div>
<details><summary>Code</summary>
<div><pre><code>::: demo A normal demo

<span><span>```</span><span>md</span>
<span><span><span>#</span> Title</span>

is very handsome.</span>
<span>```</span></span>

<span><span>```</span><span>ts</span>
<span><span>const</span> message<span>:</span> <span>string</span> <span>=</span> <span>"Mr.Hope"</span><span>;</span>

document<span>.</span><span>querySelector</span><span>(</span><span>"h1"</span><span>)</span><span>.</span>innerHTML <span>=</span> message<span>;</span></span>
<span>```</span></span>

<span><span>```</span><span>scss</span>
<span><span>h1 </span><span>{</span>
  <span>font-style</span><span>:</span> italic<span>;</span>

  <span>+ p </span><span>{</span>
    <span>color</span><span>:</span> red<span>;</span>
  <span>}</span>
<span>}</span></span>
<span>```</span></span>

:::
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br><span>22</span><br><span>23</span><br><span>24</span><br><span>25</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/v1/guide/markdown/demo/" target="_blank" rel="noopener noreferrer">View Detail</a></li>
</ul>
<h3 id="presentation"> Presentation</h3>
<i>Not supported content</i><details><summary>Code</summary>
<div><pre><code>@slidestart

<span><span>##</span> Slide 1</span>

A paragraph with some text and a <span>[<span>link</span>](<span>https://mrhope.site</span>)</span>

<span>---</span>

<span><span>##</span> Slide 2</span>

<span>-</span> Item 1
<span>-</span> Item 2

<span>---</span>

<span><span>##</span> Slide 3.1</span>

<span><span>```</span><span>js</span>
<span><span>const</span> a <span>=</span> <span>1</span><span>;</span></span>
<span>```</span></span>

--

<span><span>##</span> Slide 3.2</span>

$$
J(\theta_0,\theta_1) = \sum_{i=0}
$$

@slideend
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br><span>22</span><br><span>23</span><br><span>24</span><br><span>25</span><br><span>26</span><br><span>27</span><br><span>28</span><br><span>29</span><br><span>30</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/v1/guide/markdown/presentation/" target="_blank" rel="noopener noreferrer">View Detail</a></li>
</ul>
<h2 id="other-syntax"> Other Syntax</h2>
<div><p>custom title</p>
<p>A custom information container</p>
</div>
<div><p>custom title</p>
<p>A custom tip container</p>
</div>
<div><p>custom title</p>
<p>A custom warning container</p>
</div>
<div><p>custom Title</p>
<p>A custom danger container</p>
</div>
<details><summary>custom title</summary>
<p>A custom details container</p>
</details>
<details><summary>Code</summary>
<div><pre><code>::: info custom title

A custom information container

:::

::: tip custom title

A custom tip container

:::

::: warning custom title

A custom warning container

:::

::: danger custom Title

A custom danger container

:::

::: details custom title

A custom details container

:::
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br><span>22</span><br><span>23</span><br><span>24</span><br><span>25</span><br><span>26</span><br><span>27</span><br><span>28</span><br><span>29</span><br></div></div></details>
<hr>
<section>
<ol>
<li id="footnote1"><p>This is footnote content </p>
</li>
</ol>
</section>
]]></content:encoded>
    </item>
    <item>
      <title>Encryption article</title>
      <link>https://jiangxj.top/vuepress/vuepress/en/guide/encrypt/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/en/guide/encrypt/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Encryption article</source>
      <category>Guide</category>
      <content:encoded><![CDATA[<h1 id="encryption-article"> Encryption article</h1>
<p>The actual article content.</p>
<p>Paragraph 1 text paragraph 1 text paragraph 1 text paragraph 1 text paragraph 1 text paragraph 1 text paragraph 1 text paragraph 1 text paragraph 1 text paragraph 1 text paragraph 1 text paragraph 1 text.</p>
<p>Paragraph 2 text paragraph 2 text paragraph 2 text paragraph 2 text paragraph 2 text paragraph 2 text paragraph 2 text paragraph 2 text paragraph 2 text paragraph 2 text paragraph 2 text paragraph 2 text paragraph 2 text paragraph 2 text.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Project home</title>
      <link>https://jiangxj.top/vuepress/vuepress/en/home/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/en/home/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Project home</source>
      <content:encoded><![CDATA[<p>This is an example of a normal homepage. You can place your main content here.</p>
<p>To use this layout, you need to set <code>home: true</code> in the page front matter.</p>
<p>For related descriptions of configuration items, please see <a href="https://vuepress-theme-hope.github.io/v1/guide/layout/home.html" target="_blank" rel="noopener noreferrer">Project HomePage Layout Config</a>.</p>
]]></content:encoded>
    </item>
    <item>
      <title>Blog Home</title>
      <link>https://jiangxj.top/vuepress/vuepress/en/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/en/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Blog Home</source>
    </item>
    <item>
      <title>Slide page</title>
      <link>https://jiangxj.top/vuepress/vuepress/en/slides/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/en/slides/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Slide page</source>
      <content:encoded><![CDATA[
<i>Not supported content</i>]]></content:encoded>
      <enclosure url="https://jiangxj.top/vuepress/vuepress/logo.svg" type="image/svg+xml"/>
    </item>
    <item>
      <title>Guides</title>
      <link>https://jiangxj.top/vuepress/vuepress/en/guide/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/en/guide/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Guides</source>
      <category>Guide</category>
      <content:encoded><![CDATA[<h1 id="guides"> Guides</h1>
<ul>
<li>
<p><a href="./page.html">Page Config</a></p>
</li>
<li>
<p><a href="./markdown.html">Markdown Enhance</a></p>
</li>
<li>
<p><a href="./disable.html">Function Disable</a></p>
</li>
<li>
<p><a href="./encrypt.html">Encryption Demo</a></p>
</li>
</ul>
]]></content:encoded>
    </item>
    <item>
      <title>Custom Layout</title>
      <link>https://jiangxj.top/vuepress/vuepress/en/layout/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/en/layout/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Custom Layout</source>
      <content:encoded><![CDATA[<p>You can use slots with markdown and component support to custom page layout.</p>
<div><p>注意</p>
<p>This is just a demo, you should add styles according to your own needs.</p>


</div>
<template #page-top><p>Page top content</p>
</template><template #page-bottom><p>Page bottom content</p>
</template><template #content-top><p>Content top content</p>
</template><template #content-bottom><p>Content bottom content</p>
</template><template #navbar-start><p>Navbar start content</p>
</template><template #navbar-center><p>Navbar center content</p>
</template><template #navbar-end><p>Navbar end content</p>
</template><template #sidebar-top><p>Sidebar top content</p>
</template><template #sidebar-center><p>Sidebar center content</p>
</template><template #sidebar-bottom><p>Sidebar bottom content</p>
</template><p>For details, see <a href="https://vuepress-theme-hope.github.io/v1/guide/layout/custom.html" target="_blank" rel="noopener noreferrer">Custom layout</a>.</p>
]]></content:encoded>
    </item>
    <item>
      <title>VuePress 默认主题设置</title>
      <link>https://jiangxj.top/vuepress/vuepress/guide/configuration/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/guide/configuration/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">VuePress 默认主题设置</source>
      <description>使用vuepress构建文档博客网站的默认主题配置教程，包括navbar，sidebar，多栏分组等。</description>
      <category>guide</category>
      <content:encoded><![CDATA[<h2 id="frontmatter"> frontmatter</h2>
<p>必须在每个 .md 文件开头，通常以 yaml 格式表示。</p>
<div><pre><code><span>---</span>
<span>title</span><span>:</span> hello vuepress
<span>lang</span><span>:</span> zh<span>-</span>CN

<span>sidebarDepth</span><span>:</span> <span>2</span> 
<span>navbar</span><span>:</span> <span>false</span> 
<span>sidebar</span><span>:</span> <span>false</span>
<span>search</span><span>:</span> <span>false</span>
<span>tags</span><span>:</span>
  <span>-</span> configuration
  <span>-</span> theme
  <span>-</span> indexing
<span>prev</span><span>:</span> ./some<span>-</span>other<span>-</span>page
<span>next</span><span>:</span> <span>false</span>
<span>pageClass</span><span>:</span> custom<span>-</span>page<span>-</span>class
<span>layout</span><span>:</span> SpecialLayout
<span>---</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br></div></div><ul>
<li>title 会自动作为 sidebar 的一级链接标题(优先级低于在 config.js 中配置)。</li>
<li>sidebarDepth 见下文内嵌链接</li>
<li>navbar 用于在本页面关闭导航栏</li>
<li>sidebar 用于在本页面关闭侧边栏</li>
<li>search 用于在本页面关闭搜索框</li>
<li>tag 用于搜索(内置的搜索只搜索文章的 h2, h3 和 frontmatter 中设置的 tags)</li>
<li>prev 文章底部，上一篇文章的链接</li>
<li>next 文章底部，下一篇文章的链接</li>
<li>pageClass 为这一页自定义一个 class，用于在 .vuepress/styles/index.styl 添加自定义 css</li>
<li>layout 通常每个 .md 文件内容都是在容器 <code>&lt;div class=&quot;page&quot;&gt;</code> 中渲染的，包含侧边栏，上一篇，下一篇链接等。使用这个选项，可以设置使用特定的 vue 组件渲染该 .md 文件。(.vuepress/components/SpecialLayout.vue)</li>
</ul>
<h2 id="navbar"> navbar</h2>
<p>外部链接自动获得两个属性 target=&quot;_blank&quot; &amp; rel=&quot;noopener noreferrer&quot;。你也可以自定义。</p>
<ul>
<li>text 是显示的标题</li>
<li>ariaLabel 不知道有什么用</li>
<li>link 当然就是链接啦</li>
<li>items 用于套娃，最多两层(第一层有下拉菜单，第二层之间有分割线)。</li>
</ul>
<div><div><br><br><br><br><br><br><div>&nbsp;</div><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br></div><pre><code><span>themeConfig</span><span>:</span> <span>{</span>
    <span>logo</span><span>:</span> <span>'/assets/img/ChernoLogo.png'</span><span>,</span> <span>// 左上角的图标</span>
    <span>// 导航栏</span>
    <span>nav</span><span>:</span> <span>[</span>
      <span>{</span> <span>text</span><span>:</span> <span>'Home'</span><span>,</span> <span>link</span><span>:</span> <span>'/'</span> <span>}</span><span>,</span>
      <span>{</span> <span>text</span><span>:</span> <span>'Guide'</span><span>,</span> <span>link</span><span>:</span> <span>'/guide/'</span> <span>}</span><span>,</span>
      <span>{</span> <span>text</span><span>:</span> <span>'External'</span><span>,</span> <span>link</span><span>:</span> <span>'https://google.com'</span> <span>,</span> <span>target</span><span>:</span><span>'_self'</span><span>,</span> <span>rel</span><span>:</span><span>false</span> <span>}</span><span>,</span>
      <span>{</span>
        <span>text</span><span>:</span> <span>'Languages'</span><span>,</span>
        <span>ariaLabel</span><span>:</span> <span>'Language Menu'</span><span>,</span>
        <span>items</span><span>:</span> <span>[</span> <span>// 还可以继续套娃</span>
          <span>{</span>
            <span>text</span><span>:</span> <span>'Chinese'</span><span>,</span>
            <span>ariaLabel</span><span>:</span> <span>'Chinese Menu'</span><span>,</span> <span>// 没啥用啊，这属性</span>
            <span>items</span><span>:</span> <span>[</span>
              <span>{</span> <span>text</span><span>:</span> <span>'ah'</span><span>,</span> <span>link</span><span>:</span> <span>'https://google.com'</span> <span>}</span><span>,</span>
              <span>{</span> <span>text</span><span>:</span> <span>'nj'</span><span>,</span> <span>link</span><span>:</span> <span>'https://google.com'</span> <span>}</span><span>,</span>
            <span>]</span>
          <span>}</span><span>,</span>
          <span>{</span>
            <span>text</span><span>:</span> <span>'English'</span><span>,</span>
            <span>ariaLabel</span><span>:</span> <span>'English Menu'</span><span>,</span> <span>// 没啥用啊，这属性</span>
            <span>items</span><span>:</span> <span>[</span>
              <span>{</span> <span>text</span><span>:</span> <span>'sf'</span><span>,</span> <span>link</span><span>:</span> <span>'https://google.com'</span> <span>}</span><span>,</span>
              <span>{</span> <span>text</span><span>:</span> <span>'ny'</span><span>,</span> <span>link</span><span>:</span> <span>'https://google.com'</span> <span>}</span><span>,</span>
            <span>]</span>
          <span>}</span><span>,</span>
        <span>]</span>
      <span>}</span>
    <span>]</span><span>,</span>

</code></pre><div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br><span>22</span><br><span>23</span><br><span>24</span><br><span>25</span><br><span>26</span><br><span>27</span><br><span>28</span><br><span>29</span><br><span>30</span><br><span>31</span><br></div></div><h2 id="sidebar"> sidebar</h2>
<div><pre><code><span>// .vuepress/config.js</span>
module<span>.</span>exports <span>=</span> <span>{</span>
  <span>themeConfig</span><span>:</span> <span>{</span>
    <span>sidebar</span><span>:</span> <span>[</span>
      <span>'/'</span><span>,</span>
      <span>'/page-a'</span><span>,</span>
      <span>[</span><span>'/page-b'</span><span>,</span> <span>'显示指定链接标题'</span><span>]</span>
    <span>]</span>
  <span>}</span>
<span>}</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br></div></div><p>指定路径时，可以忽略.md后缀，以'path/'结尾的路径，对应的内容为 path/README.md。如果需要显示指定标题，则如例子中的第三行。</p>
<h3 id="内嵌链接"> 内嵌链接</h3>
<p>默认抓取 Markdown 文档的所有二级标题作为当前链接的子链接。对应的设置为 themeConfig.sidebarDepth = 1。</p>
<p>sidebarDepth = 0 表示没有子链接。
sidebarDepth = 2 会抓取h2,h3作为子链接。最大层次就是2。效果见本文。</p>
<p>这个设置除了在配置中改，也可以在 .md 的 frontmatter 部分改。</p>
<div><pre><code><span>---</span>
<span>sidebarDepth</span><span>:</span> <span>0</span>
<span>---</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><h3 id="展开所有子链接"> 展开所有子链接</h3>
<div><pre><code><span>// .vuepress/config.js</span>
module<span>.</span>exports <span>=</span> <span>{</span>
  <span>themeConfig</span><span>:</span> <span>{</span>
    <span>displayAllHeaders</span><span>:</span> <span>true</span> <span>// Default: false</span>
  <span>}</span>
<span>}</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br></div></div><h3 id="导航栏分组"> 导航栏分组</h3>
<p>顾名思义</p>
<div><pre><code><span>sidebar</span><span>:</span> <span>[</span>
      <span>{</span>
        <span>title</span><span>:</span> <span>'Group1'</span><span>,</span> <span>// 必须要有分组名</span>
        <span>collapsable</span><span>:</span> <span>false</span><span>,</span> <span>// 默认是折叠，设为 true 表示直接展开</span>
        <span>children</span><span>:</span> <span>[</span><span>'/blog/'</span><span>,</span> <span>'/blog/test'</span><span>]</span> <span>// 有多个分组成员</span>
      <span>}</span><span>,</span>
      <span>{</span> <span>// 只有一个分组成员</span>
        <span>title</span><span>:</span> <span>'Group2'</span><span>,</span>
        <span>path</span><span>:</span> <span>'/guide/'</span><span>,</span>
      <span>}</span><span>,</span>
      <span>'/'</span><span>,</span>
    <span>]</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br></div></div><h3 id="多重侧边栏"> 多重侧边栏</h3>
<p>不同的分区可以对应不同的侧边导航栏。<br>
<strong>fallback 必须放在最后！</strong><br>
auto 的意思是这一个 md 自动生成一个它独占的侧边栏。</p>
<div><div><br><br><br><br><br><br><br><br><br><br><br><br><br><div>&nbsp;</div><br><br><br><br><br><br><br><br><br></div><pre><code><span>sidebar</span><span>:</span> <span>{</span>
      <span>'/foo/'</span><span>:</span> <span>[</span>
        <span>''</span><span>,</span>     <span>/* /foo/ */</span>
        <span>'one'</span><span>,</span>  <span>/* /foo/one.html */</span>
        <span>'two'</span>   <span>/* /foo/two.html */</span>
      <span>]</span><span>,</span>

      <span>'/bar/'</span><span>:</span> <span>[</span>
        <span>''</span><span>,</span>      <span>/* /bar/ */</span>
        <span>'three'</span><span>,</span> <span>/* /bar/three.html */</span>
        <span>'four'</span>   <span>/* /bar/four.html */</span>
      <span>]</span><span>,</span>

      <span>'/baz/'</span><span>:</span> <span>'auto'</span><span>,</span> <span>/* automatically generate single-page sidebars */</span>

      <span>// fallback</span>
      <span>'/'</span><span>:</span> <span>[</span>
        <span>''</span><span>,</span>        <span>/* / */</span>
        <span>'contact'</span><span>,</span> <span>/* /contact.html */</span>
        <span>'about'</span>    <span>/* /about.html */</span>
      <span>]</span>
    <span>}</span>
</code></pre><div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br><span>22</span><br></div></div><h2 id="code-groups"> code groups</h2>
<p>CodeGroupItem之间需要空一行</p>
<div><pre><code><span><span><span>&lt;</span>CodeGroup</span><span>></span></span>
<span><span><span>&lt;</span>CodeGroupItem</span> <span>title</span><span><span>=</span><span>"</span>YARN<span>"</span></span><span>></span></span>
\`\`\`bash
yarn create vuepress-site [optionalDirectoryName]
\`\`\`
<span><span><span>&lt;/</span>CodeGroupItem</span><span>></span></span>

<span><span><span>&lt;</span>CodeGroupItem</span> <span>title</span><span><span>=</span><span>"</span>NPM<span>"</span></span><span>></span></span>
\`\`\`bash
npx create-vuepress-site [optionalDirectoryName]
\`\`\`
<span><span><span>&lt;/</span>CodeGroupItem</span><span>></span></span>
<span><span><span>&lt;/</span>CodeGroup</span><span>></span></span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br></div></div><p>效果如下</p>
<CodeGroup>
<CodeGroupItem title="YARN">
<div><pre><code><span>yarn</span> create vuepress-site <span>[</span>optionalDirectoryName<span>]</span>
</code></pre>
<div><span>1</span><br></div></div></CodeGroupItem>
<CodeGroupItem title="NPM">
<div><pre><code>npx create-vuepress-site <span>[</span>optionalDirectoryName<span>]</span>
</code></pre>
<div><span>1</span><br></div></div></CodeGroupItem>
</CodeGroup>]]></content:encoded>
    </item>
    <item>
      <title>组件禁用</title>
      <link>https://jiangxj.top/vuepress/vuepress/guide/disable/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/guide/disable/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">组件禁用</source>
      <description>&amp;lt;p&amp;gt;你可以通过设置页面的 Frontmatter，在页面禁用一些功能。&amp;lt;/p&amp;gt;
</description>
      <category>使用指南</category>
      <content:encoded><![CDATA[<p>你可以通过设置页面的 Frontmatter，在页面禁用一些功能。</p>

<p>本页面应当禁用了:</p>
<ul>
<li>导航栏</li>
<li>侧边栏</li>
<li>路径导航</li>
<li>页面信息</li>
<li>贡献者</li>
<li>编辑此页链接</li>
<li>更新时间</li>
<li>上一篇/下一篇 链接</li>
<li>评论</li>
<li>页脚</li>
<li>返回顶部按钮</li>
</ul>
]]></content:encoded>
    </item>
    <item>
      <title>密码加密的文章</title>
      <link>https://jiangxj.top/vuepress/vuepress/guide/encrypt/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/guide/encrypt/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">密码加密的文章</source>
      <category>使用指南</category>
      <content:encoded><![CDATA[<h1 id="密码加密的文章"> 密码加密的文章</h1>
<p>实际的文章内容。</p>
<p>段落 1 文字段落 1 文字段落 1 文字段落 1 文字段落 1 文字段落 1 文字段落 1 文字段落 1 文字段落 1 文字段落 1 文字段落 1 文字段落 1 文字。</p>
<p>段落 2 文字段落 2 文字段落 2 文字段落 2 文字段落 2 文字段落 2 文字段落 2 文字段落 2 文字段落 2 文字段落 2 文字段落 2 文字段落 2 文字段落 2 文字段落 2 文字。</p>
]]></content:encoded>
    </item>
    <item>
      <title>Markdown 增强</title>
      <link>https://jiangxj.top/vuepress/vuepress/guide/markdown/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/guide/markdown/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Markdown 增强</source>
      <description>关于Markdown的基础使用教程介绍。</description>
      <category>使用指南</category>
      <content:encoded><![CDATA[<p><code>vuepress-theme-hope</code> 通过内置 <a href="https://vuepress-theme-hope.github.io/md-enhance" target="_blank" rel="noopener noreferrer">md-enhance</a>，在 Markdown 中启用了更多的语法与新功能。</p>

<h2 id="一键启用"> 一键启用</h2>
<p>你可以设置 <code>themeconfig.mdEnhance.enableAll</code> 启用 <a href="https://vuepress-theme-hope.github.io/md-enhance" target="_blank" rel="noopener noreferrer">md-enhance</a> 插件的所有功能。</p>
<div><div><br><br><div>&nbsp;</div><div>&nbsp;</div><div>&nbsp;</div><br><br><br></div><pre><code>module<span>.</span>exports <span>=</span> <span>{</span>
  <span>themeConfig</span><span>:</span> <span>{</span>
    <span>mdEnhance</span><span>:</span> <span>{</span>
      <span>enableAll</span><span>:</span> <span>true</span><span>,</span>
    <span>}</span><span>,</span>
  <span>}</span><span>,</span>
<span>}</span><span>;</span>
</code></pre><div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br></div></div><h2 id="新增的更多语法"> 新增的更多语法</h2>
<h3 id="上下角标"> 上下角标</h3>
<p>19<sup>th</sup> H<sub>2</sub>O</p>
<details><summary>代码</summary>
<div><pre><code>19^th^ H<span><span>~</span><span>2</span><span>~</span></span>O
</code></pre>
<div><span>1</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/zh/guide/markdown/sup-sub/" target="_blank" rel="noopener noreferrer">点击查看</a></li>
</ul>
<h3 id="自定义对齐"> 自定义对齐</h3>
<p>::: center</p>
<p>我是居中的</p>
<p>:::</p>
<p>::: right</p>
<p>我在右对齐</p>
<p>:::</p>
<details><summary>代码</summary>
<div><pre><code>::: center

我是居中的

:::

::: right

我在右对齐

:::
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/zh/guide/markdown/align/" target="_blank" rel="noopener noreferrer">点击查看</a></li>
</ul>
<h3 id="脚注"> 脚注</h3>
<p>此文字有脚注<sup></sup>.</p>
<details><summary>代码</summary>
<div><pre><code>此文字有脚注[^first].

<span><span>[</span><span>^first</span><span>]</span><span>:</span> 这是脚注内容</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/zh/guide/markdown/footnote/" target="_blank" rel="noopener noreferrer">点击查看</a></li>
</ul>
<h3 id="标记"> 标记</h3>
<p>你可以标记 ==重要的内容== 。</p>
<details><summary>代码</summary>
<div><pre><code>你可以标记 ==重要的内容== 。
</code></pre>
<div><span>1</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/zh/guide/markdown/mark/" target="_blank" rel="noopener noreferrer">点击查看</a></li>
</ul>
<h3 id="任务列表"> 任务列表</h3>
<ul>
<li>[x] 计划 1</li>
<li>[ ] 计划 2</li>
</ul>
<details><summary>Code</summary>
<div><pre><code><span>-</span> [x] 计划 1
<span>-</span> [ ] 计划 2
</code></pre>
<div><span>1</span><br><span>2</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/guide/markdown/tasklist/" target="_blank" rel="noopener noreferrer">点击查看</a></li>
</ul>
<h3 id="流程图"> 流程图</h3>
<i>Not supported content</i><details><summary>代码</summary>
<div><pre><code><span><span>```</span><span>flow</span>
<span>cond=>condition: Process?
process=>operation: Process
e=>end: End

cond(yes)->process->e
cond(no)->e</span>
<span>```</span></span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/zh/guide/markdown/flowchart/" target="_blank" rel="noopener noreferrer">点击查看</a></li>
</ul>
<h2 id="mermaid"> Mermaid</h2>
<Mermaid id="mermaid-64a57026" data-code="sequenceDiagram%0A%20%20%20%20Alice%20-%3E%3E%20Bob%3A%20Hello%20Bob%2C%20how%20are%20you%3F%0A%20%20%20%20Bob--%3E%3EJohn%3A%20How%20about%20you%20John%3F%0A%20%20%20%20Bob--x%20Alice%3A%20I%20am%20good%20thanks!%0A%20%20%20%20Bob-x%20John%3A%20I%20am%20good%20thanks!%0A%20%20%20%20Note%20right%20of%20John%3A%20Bob%20thinks%20a%20long%3Cbr%2F%3Elong%20time%2C%20so%20long%3Cbr%2F%3Ethat%20the%20text%20does%3Cbr%2F%3Enot%20fit%20on%20a%20row.%0A%0A%20%20%20%20Bob--%3EAlice%3A%20Checking%20with%20John...%0A%20%20%20%20Alice-%3EJohn%3A%20Yes...%20John%2C%20how%20are%20you%3F%0A"></Mermaid><details><summary>代码</summary>
<div><pre><code>```sequence
Alice ->> Bob: Hello Bob, how are you?
Bob-->>John: How about you John?
Bob--x Alice: I am good thanks!
Bob-x John: I am good thanks!
Note right of John: Bob thinks a long<span><span><span>&lt;</span>br</span><span>/></span></span>long time, so long<span><span><span>&lt;</span>br</span><span>/></span></span>that the text does<span><span><span>&lt;</span>br</span><span>/></span></span>not fit on a row.

Bob-->Alice: Checking with John...
Alice->John: Yes... John, how are you?
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br></div></div><div><pre><code>
</code></pre>
<div><span>1</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/zh/guide/markdown/mermaid/" target="_blank" rel="noopener noreferrer">点击查看</a></li>
</ul>
<h3 id="tex-语法"> Tex 语法</h3>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:2.4em;vertical-align:-0.95em;"></span><span><span></span><span><span><span><span style="height:1.3714em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span style="margin-right:0.05556em;">∂</span><span><span style="margin-right:0.03588em;">ω</span><span><span><span><span style="height:0.5904em;"><span style="top:-2.989em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.02778em;">r</span></span></span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.05556em;">∂</span><span><span><span><span style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.02778em;">r</span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;"><span>(</span></span><span><span></span><span><span><span><span style="height:1.3414em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span style="margin-right:0.03588em;">ω</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span style="margin-right:0.03588em;">ω</span></span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="top:0em;"><span>)</span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:3.0277em;vertical-align:-1.2777em;"></span><span><span style="top:0em;"><span>(</span></span><span><span></span><span><span><span><span style="height:1.3414em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span style="margin-right:0.03588em;">ω</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span style="margin-right:0.03588em;">ω</span></span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="top:0em;"><span>)</span></span></span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;"><span>{</span></span><span>(</span><span>lo<span style="margin-right:0.01389em;">g</span></span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span><span>)</span><span><span><span><span style="height:0.7144em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.02778em;">r</span></span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span style="margin-right:0.02778em;">r</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span></span><span><span><span><span style="height:1.5017em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.03588em;">ω</span><span><span><span><span style="height:0.7507em;"><span style="top:-2.989em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>(</span><span>−</span><span>1</span><span><span>)</span><span><span><span><span style="height:0.8247em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span></span></span></span></span><span style="margin-right:0.02778em;">r</span><span style="margin-right:0.1667em;"></span><span>⋯</span><span style="margin-right:0.1667em;"></span><span>(</span><span style="margin-right:0.02778em;">r</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span>i</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span>1</span><span>)</span><span>(</span><span>lo<span style="margin-right:0.01389em;">g</span></span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span><span>)</span><span><span><span><span style="height:0.8247em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span style="margin-right:0.02778em;">r</span><span>−</span><span>i</span></span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="top:0em;"><span>}</span></span></span></span></span></span></span></p>
<details><summary>代码</summary>
<div><pre><code>$$
\frac {\partial^r} {\partial \omega^r} \left(\frac {y^{\omega}} {\omega}\right)
= \left(\frac {y^{\omega}} {\omega}\right) \left\{(\log y)^r + \sum_{i=1}^r \frac {(-1)^i r \cdots (r-i+1) (\log y)^{r-i}} {\omega^i} \right\}
$$
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/zh/guide/markdown/tex/" target="_blank" rel="noopener noreferrer">点击查看</a></li>
</ul>
<h3 id="代码案例"> 代码案例</h3>

          <div
            id="code-demo-5ac6bc9f"
           
  
 data-title="%E4%B8%80%E4%B8%AA%E6%99%AE%E9%80%9A%20Demo"

            data-code="%7B%22html%22%3A%22%3Ch1%3EMr.Hope%3C%2Fh1%3E%5Cn%3Cp%3E%3Cspan%20id%3D%5C%22very%5C%22%3E%E5%8D%81%E5%88%86%3C%2Fspan%3E%20%E5%B8%85%3C%2Fp%3E%5Cn%22%2C%22js%22%3A%22document.querySelector(%5C%22%23very%5C%22).addEventListener(%5C%22click%5C%22%2C%20()%20%3D%3E%20%7B%5Cn%20%20alert(%5C%22%E5%8D%81%E5%88%86%E5%B8%85%5C%22)%3B%5Cn%7D)%3B%5Cn%22%2C%22css%22%3A%22span%20%7B%5Cn%20%20color%3A%20red%3B%5Cn%7D%5Cn%22%7D"
          >
              
              <div>
                <div>
<div><pre><code><span><span><span>&lt;</span>h1</span><span>></span></span>Mr.Hope<span><span><span>&lt;/</span>h1</span><span>></span></span>
<span><span><span>&lt;</span>p</span><span>></span></span><span><span><span>&lt;</span>span</span> <span>id</span><span><span>=</span><span>"</span>very<span>"</span></span><span>></span></span>十分<span><span><span>&lt;/</span>span</span><span>></span></span> 帅<span><span><span>&lt;/</span>p</span><span>></span></span>
</code></pre>
<div><span>1</span><br><span>2</span><br></div></div><div><pre><code>document<span>.</span><span>querySelector</span><span>(</span><span>"#very"</span><span>)</span><span>.</span><span>addEventListener</span><span>(</span><span>"click"</span><span>,</span> <span>(</span><span>)</span> <span>=></span> <span>{</span>
  <span>alert</span><span>(</span><span>"十分帅"</span><span>)</span><span>;</span>
<span>}</span><span>)</span><span>;</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><div><pre><code><span>span</span> <span>{</span>
  <span>color</span><span>:</span> red<span>;</span>
<span>}</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div>
            </div>
          </div>
          
        </div>
<details><summary>代码</summary>
<div><pre><code>::: demo 一个普通 Demo

<span><span>```</span><span>html</span>
<span><span><span><span>&lt;</span>h1</span><span>></span></span>Mr.Hope<span><span><span>&lt;/</span>h1</span><span>></span></span>
<span><span><span>&lt;</span>p</span><span>></span></span><span><span><span>&lt;</span>span</span> <span>id</span><span><span>=</span><span>"</span>very<span>"</span></span><span>></span></span>十分<span><span><span>&lt;/</span>span</span><span>></span></span> 帅<span><span><span>&lt;/</span>p</span><span>></span></span></span>
<span>```</span></span>

<span><span>```</span><span>js</span>
<span>document<span>.</span><span>querySelector</span><span>(</span><span>"#very"</span><span>)</span><span>.</span><span>addEventListener</span><span>(</span><span>"click"</span><span>,</span> <span>(</span><span>)</span> <span>=></span> <span>{</span>
  <span>alert</span><span>(</span><span>"十分帅"</span><span>)</span><span>;</span>
<span>}</span><span>)</span><span>;</span></span>
<span>```</span></span>

<span><span>```</span><span>css</span>
<span><span>span</span> <span>{</span>
  <span>color</span><span>:</span> red<span>;</span>
<span>}</span></span>
<span>```</span></span>

:::
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br></div></div></details>

          <div
            id="code-demo-693e1878"
           
  data-type="react"
 data-title="%E4%B8%80%E4%B8%AA%20React%20Demo"

            data-code="%7B%22js%22%3A%22export%20default%20class%20App%20extends%20React.Component%20%7B%5Cn%20%20constructor(props)%20%7B%5Cn%20%20%20%20super(props)%3B%5Cn%20%20%20%20this.state%20%3D%20%7B%20message%3A%20%5C%22%E5%8D%81%E5%88%86%E5%B8%85%5C%22%20%7D%3B%5Cn%20%20%7D%5Cn%20%20render()%20%7B%5Cn%20%20%20%20return%20(%5Cn%20%20%20%20%20%20%3Cdiv%20className%3D%5C%22box-react%5C%22%3E%5Cn%20%20%20%20%20%20%20%20Mr.Hope%20%3Cspan%3E%7Bthis.state.message%7D%3C%2Fspan%3E%5Cn%20%20%20%20%20%20%3C%2Fdiv%3E%5Cn%20%20%20%20)%3B%5Cn%20%20%7D%5Cn%7D%5Cn%22%2C%22css%22%3A%22.box-react%20span%20%7B%5Cn%20%20color%3A%20red%3B%5Cn%7D%5Cn%22%7D"
          >
              
              <div>
                <div>
<div><pre><code><span>export</span> <span>default</span> <span>class</span> <span>App</span> <span>extends</span> <span>React<span>.</span>Component</span> <span>{</span>
  <span>constructor</span><span>(</span><span>props</span><span>)</span> <span>{</span>
    <span>super</span><span>(</span>props<span>)</span><span>;</span>
    <span>this</span><span>.</span>state <span>=</span> <span>{</span> <span>message</span><span>:</span> <span>"十分帅"</span> <span>}</span><span>;</span>
  <span>}</span>
  <span>render</span><span>(</span><span>)</span> <span>{</span>
    <span>return</span> <span>(</span>
      <span>&lt;</span>div className<span>=</span><span>"box-react"</span><span>></span>
        Mr<span>.</span>Hope <span>&lt;</span>span<span>></span><span>{</span><span>this</span><span>.</span>state<span>.</span>message<span>}</span><span>&lt;</span><span>/</span>span<span>></span>
      <span>&lt;</span><span>/</span>div<span>></span>
    <span>)</span><span>;</span>
  <span>}</span>
<span>}</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br></div></div><div><pre><code><span>.box-react span</span> <span>{</span>
  <span>color</span><span>:</span> red<span>;</span>
<span>}</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div>
            </div>
          </div>
          
        </div>
<details><summary>代码</summary>
<div><pre><code>::: demo [react] 一个 React Demo

<span><span>```</span><span>js</span>
<span><span>export</span> <span>default</span> <span>class</span> <span>App</span> <span>extends</span> <span>React<span>.</span>Component</span> <span>{</span>
  <span>constructor</span><span>(</span><span>props</span><span>)</span> <span>{</span>
    <span>super</span><span>(</span>props<span>)</span><span>;</span>
    <span>this</span><span>.</span>state <span>=</span> <span>{</span> <span>message</span><span>:</span> <span>"十分帅"</span> <span>}</span><span>;</span>
  <span>}</span>
  <span>render</span><span>(</span><span>)</span> <span>{</span>
    <span>return</span> <span>(</span>
      <span>&lt;</span>div className<span>=</span><span>"box-react"</span><span>></span>
        Mr<span>.</span>Hope <span>&lt;</span>span<span>></span><span>{</span><span>this</span><span>.</span>state<span>.</span>message<span>}</span><span>&lt;</span><span>/</span>span<span>></span>
      <span>&lt;</span><span>/</span>div<span>></span>
    <span>)</span><span>;</span>
  <span>}</span>
<span>}</span></span>
<span>```</span></span>

<span><span>```</span><span>css</span>
<span><span>.box-react span</span> <span>{</span>
  <span>color</span><span>:</span> red<span>;</span>
<span>}</span></span>
<span>```</span></span>

:::
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br><span>22</span><br><span>23</span><br><span>24</span><br><span>25</span><br></div></div></details>

          <div
            id="code-demo-2c0b9cdb"
           
  data-type="vue"
 data-title="%E4%B8%80%E4%B8%AA%20Vue%20Demo"

            data-code="%7B%22vue%22%3A%22%3Ctemplate%3E%5Cn%20%20%3Cdiv%20class%3D%5C%22box%5C%22%3E%5Cn%20%20%20%20Mr.Hope%20%3Cspan%3E%7B%7B%20message%20%7D%7D%3C%2Fspan%3E%5Cn%20%20%3C%2Fdiv%3E%5Cn%3C%2Ftemplate%3E%5Cn%3Cscript%3E%5Cnexport%20default%20%7B%5Cn%20%20data%3A%20()%20%3D%3E%20(%7B%20message%3A%20%5C%22%E5%8D%81%E5%88%86%E5%B8%85%5C%22%20%7D)%2C%5Cn%7D%3B%5Cn%3C%2Fscript%3E%5Cn%3Cstyle%3E%5Cn.box%20span%20%7B%5Cn%20%20color%3A%20red%3B%5Cn%7D%5Cn%3C%2Fstyle%3E%5Cn%22%7D"
          >
              
              <div>
                <div>
<div><pre><code><span><span><span>&lt;</span>template</span><span>></span></span>
  <span><span><span>&lt;</span>div</span> <span>class</span><span><span>=</span><span>"</span>box<span>"</span></span><span>></span></span>
    Mr.Hope <span><span><span>&lt;</span>span</span><span>></span></span>{{ message }}<span><span><span>&lt;/</span>span</span><span>></span></span>
  <span><span><span>&lt;/</span>div</span><span>></span></span>
<span><span><span>&lt;/</span>template</span><span>></span></span>
<span><span><span>&lt;</span>script</span><span>></span></span><span><span>
<span>export</span> <span>default</span> <span>{</span>
  <span>data</span><span>:</span> <span>(</span><span>)</span> <span>=></span> <span>(</span><span>{</span> <span>message</span><span>:</span> <span>"十分帅"</span> <span>}</span><span>)</span><span>,</span>
<span>}</span><span>;</span>
</span></span><span><span><span>&lt;/</span>script</span><span>></span></span>
<span><span><span>&lt;</span>style</span><span>></span></span><span><span>
<span>.box span</span> <span>{</span>
  <span>color</span><span>:</span> red<span>;</span>
<span>}</span>
</span></span><span><span><span>&lt;/</span>style</span><span>></span></span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br></div></div>
            </div>
          </div>
          
        </div>
<details><summary>代码</summary>
<div><pre><code>::: demo [vue] 一个 Vue Demo

<span><span>```</span><span>vue</span>
<span>&lt;template>
  &lt;div>
    Mr.Hope &lt;span>{{ message }}&lt;/span>
  &lt;/div>
&lt;/template>
&lt;script>
export default {
  data: () => ({ message: "十分帅" }),
};
&lt;/script>
&lt;style>
.box span {
  color: red;
}
&lt;/style></span>
<span>```</span></span>

:::
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br></div></div></details>

          <div
            id="code-demo-2869b0e2"
           
  
 data-title="%E4%B8%80%E4%B8%AA%E6%99%AE%E9%80%9A%20Demo"

            data-code="%7B%22md%22%3A%22%23%20%E6%A0%87%E9%A2%98%5Cn%5Cn%E5%8D%81%E5%88%86%E5%B8%85%5Cn%22%2C%22ts%22%3A%22const%20message%3A%20string%20%3D%20%5C%22Mr.Hope%5C%22%3B%5Cn%5Cndocument.querySelector(%5C%22h1%5C%22).innerHTML%20%3D%20message%3B%5Cn%22%2C%22scss%22%3A%22h1%20%7B%5Cn%20%20font-style%3A%20italic%3B%5Cn%5Cn%20%20%2B%20p%20%7B%5Cn%20%20%20%20color%3A%20red%3B%5Cn%20%20%7D%5Cn%7D%5Cn%22%7D"
          >
              
              <div>
                <div>
<div><pre><code><span><span>#</span> 标题</span>

十分帅
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><div><pre><code><span>const</span> message<span>:</span> <span>string</span> <span>=</span> <span>"Mr.Hope"</span><span>;</span>

document<span>.</span><span>querySelector</span><span>(</span><span>"h1"</span><span>)</span><span>.</span>innerHTML <span>=</span> message<span>;</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><div><pre><code><span>h1 </span><span>{</span>
  <span>font-style</span><span>:</span> italic<span>;</span>

  <span>+ p </span><span>{</span>
    <span>color</span><span>:</span> red<span>;</span>
  <span>}</span>
<span>}</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br></div></div>
            </div>
          </div>
          
        </div>
<details><summary>代码</summary>
<div><pre><code>::: demo 一个普通 Demo

<span><span>```</span><span>md</span>
<span><span><span>#</span> 标题</span>

十分帅</span>
<span>```</span></span>

<span><span>```</span><span>ts</span>
<span><span>const</span> message<span>:</span> <span>string</span> <span>=</span> <span>"Mr.Hope"</span><span>;</span>

document<span>.</span><span>querySelector</span><span>(</span><span>"h1"</span><span>)</span><span>.</span>innerHTML <span>=</span> message<span>;</span></span>
<span>```</span></span>

<span><span>```</span><span>scss</span>
<span><span>h1 </span><span>{</span>
  <span>font-style</span><span>:</span> italic<span>;</span>

  <span>+ p </span><span>{</span>
    <span>color</span><span>:</span> red<span>;</span>
  <span>}</span>
<span>}</span></span>
<span>```</span></span>

:::
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br><span>22</span><br><span>23</span><br><span>24</span><br><span>25</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/zh/guide/markdown/demo/" target="_blank" rel="noopener noreferrer">点击查看</a></li>
</ul>
<h3 id="幻灯片"> 幻灯片</h3>
<i>Not supported content</i><details><summary>代码</summary>
<div><pre><code>@slidestart

<span><span>##</span> 幻灯片 1</span>

一个有文字和 <span>[<span>链接</span>](<span>https://mrhope.site</span>)</span> 的段落

<span>---</span>

<span><span>##</span> 幻灯片 2</span>

<span>-</span> 列表 1
<span>-</span> 列表 2

<span>---</span>

<span><span>##</span> 幻灯片 3.1</span>

<span><span>```</span><span>js</span>
<span><span>const</span> a <span>=</span> <span>1</span><span>;</span></span>
<span>```</span></span>

--

<span><span>##</span> 幻灯片 3.2</span>

$$
J(\theta_0,\theta_1) = \sum_{i=0}
$$

@slideend
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br><span>22</span><br><span>23</span><br><span>24</span><br><span>25</span><br><span>26</span><br><span>27</span><br><span>28</span><br><span>29</span><br><span>30</span><br></div></div></details>
<ul>
<li><a href="https://vuepress-theme-hope.github.io/zh/guide/markdown/presentation/" target="_blank" rel="noopener noreferrer">点击查看</a></li>
</ul>
<h2 id="其他语法"> 其他语法</h2>
<div><p>自定义标题</p>
<p>信息容器</p>
</div>
<div><p>自定义标题</p>
<p>提示容器</p>
</div>
<div><p>自定义标题</p>
<p>警告容器</p>
</div>
<div><p>自定义标题</p>
<p>危险容器</p>
</div>
<details><summary>自定义标题</summary>
<p>详情容器</p>
</details>
<details><summary>代码</summary>
<div><pre><code>::: info 自定义标题

信息容器

:::

::: tip 自定义标题

提示容器

:::

::: warning 自定义标题

警告容器

:::

::: danger 自定义标题

危险容器

:::

::: details 自定义标题

详情容器

:::
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br><span>22</span><br><span>23</span><br><span>24</span><br><span>25</span><br><span>26</span><br><span>27</span><br><span>28</span><br><span>29</span><br></div></div></details>
<hr>
<section>
<ol>
<li id="footnote1"><p>这是脚注内容 </p>
</li>
</ol>
</section>
]]></content:encoded>
    </item>
    <item>
      <title>页面配置</title>
      <link>https://jiangxj.top/vuepress/vuepress/guide/page/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/guide/page/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">页面配置</source>
      <description>在Markdown中的一些细节问题总结。</description>
      <category>guide</category>
      <pubDate>Thu, 03 Mar 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="注意问题"> 注意问题</h2>
<p>（1）文章内容按照2-3级标题排版</p>
<p>（2）插入图片不支持<code>.webp</code>格式</p>
<h2 id="自动部署"> 自动部署</h2>
<p>创建<code>deploy.sh</code>文件，内容如下：</p>
<div><pre><code>#!/usr/bin/env sh
#确保脚本抛出遇到的错误
set -e
#生成静态文件
yarn docs:build
#进入生成的文件夹
cd ./dist
git init
#把当前目录及子目录变动都加到暂存区
git add .
#把项目中所有文件的变动都加到暂存区
#git add -a
git commit -m &quot;deploy&quot;
git push -f https://github.com/jiangxjun/vuepress.git
cd -
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br></div></div><h2 id="页面信息"> 页面信息</h2>
<p>你可以在 Markdown 的 Frontmatter 中设置页面信息。</p>
<ul>
<li>
<p>作者设置为 #XJ。</p>
</li>
<li>
<p>写作时间应为 2022 年 3 月 3 日</p>
</li>
<li>
<p>分类为 “使用指南”</p>
</li>
<li>
<p>标签为 “页面配置” 和 “使用指南”</p>
</li>
</ul>
<h2 id="页面内容"> 页面内容</h2>
<p>你可以自由在这里书写你的 Markdown。</p>
<div><p>提示</p>
<ul>
<li>
<p>Markdown 文件夹的图片请使用相对链接 <code>./</code> 进行引用。</p>
</li>
<li>
<p><code>.vuepress/public</code> 文件夹的图片，请使用绝对链接 <code>/</code> 进行引用</p>
</li>
</ul>
</div>
<p>主题包含了一个自定义徽章章可以使用:</p>
<blockquote>
<p>文字结尾应该有深蓝色的 徽章文字 徽章。 <i>Not supported content</i></p>
</blockquote>
<h2 id="页面结构"> 页面结构</h2>
<p>此页面应当包含：</p>
<ul>
<li>返回顶部按钮</li>
<li>路径导航</li>
<li>评论</li>
<li>页脚</li>
</ul>
<h2 id="slide"> slide</h2>
<i>Not supported content</i>]]></content:encoded>
    </item>
    <item>
      <title>Intro Page</title>
      <link>https://jiangxj.top/vuepress/vuepress/intro/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/intro/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Intro Page</source>
      <content:encoded><![CDATA[<h1 id="intro-page"> Intro Page</h1>
<p>Place your introducation and profile here.</p>
]]></content:encoded>
    </item>
    <item>
      <title>这主题废弃了标题？</title>
      <link>https://jiangxj.top/vuepress/vuepress/home/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/home/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">这主题废弃了标题？</source>
      <content:encoded><![CDATA[<p>This is an example of a normal homepage. You can place your main content here.</p>
<p>To use this layout, you need to set <code>home: true</code> in the page front matter.</p>
<p>For related descriptions of configuration items, please see <a href="https://vuepress-theme-hope.github.io/guide/layout/home/" target="_blank" rel="noopener noreferrer">Project HomePage Layout Config</a>.</p>
]]></content:encoded>
    </item>
    <item>
      <title>深度强化学习——基础、研究及应用</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/DRL/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/DRL/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">深度强化学习——基础、研究及应用</source>
      <description>基于北京大学前沿计算研究中心助理教授董豪博士等编写了一本名为《Deep Reinforcement Learning——Fundamentals, Research and Applications》一书的学习整理。</description>
      <content:encoded><![CDATA[<p>北京大学前沿计算研究中心助理教授董豪博士等编写了一本名为*《Deep Reinforcement Learning:  Fundamentals, Research and Applications》*的英文书籍。该书于 2020 年 6 月由 Springer 发行。</p>
<center><img src=".\images\coverpage.png" width=50%></center>
<p><strong>本书网址</strong>：https://deepreinforcementlearningbook.org/#resources</p>
<blockquote>
<p>Hao Dong, Zihan Ding, Shanghang Zhang, Hang Yuan, Hongming Zhang, Jingqing Zhang, Yanhua Huang, Tianyang Yu, Huaqing Zhang, Ruitong Huang. (2020) Deep Reinforcement Learning: Fundamentals, Research, and Applications. Springer.</p>
</blockquote>
<p><strong>本书开源代码</strong>：</p>
<blockquote>
<p><a href="https://github.com/tensorlayer/tensorlayer/tree/master/examples/reinforcement_learning" target="_blank" rel="noopener noreferrer">Code for Fundamental Chapters</a></p>
<p><a href="https://github.com/deep-reinforcement-learning-book" target="_blank" rel="noopener noreferrer">Code for Application Chapters</a></p>
<p><a href="https://github.com/tensorlayer/RLzoo" target="_blank" rel="noopener noreferrer">RL Zoo for Industry</a></p>
</blockquote>
<p>深度强化学习（DRL）结合深度学习（DL）和强化学习（RL）算法的各自优势，用于解决复杂的决策问题（decision-making tasks）。得益于DeepMind AlphaGo和OpenAI Five等成功案例，深度强化学习受到大量的研究和关注，相关技术广泛应用于不同的领域。</p>
<p>全书分为三部分：</p>
<p><strong>第一部分（基础部分）介绍了强化学习的基础知识、常用的深度强化学习算法及其实现方法。</strong></p>
<center><img src=".\images\part11.png" width=70%></center>
<center><img src=".\images\part12.png" width=70%></center>
<center><img src=".\images\part13.png" width=70%></center>
<center><img src=".\images\part14.png" width=70%></center>
<center><img src=".\images\part15.png" width=70%></center>
<center><img src=".\images\part16.png" width=70%></center>
<p><strong>第二部分：关于深度强化学习研究方向的一些介绍</strong></p>
<center><img src=".\images\part21.png" width=70%></center>
<center><img src=".\images\part22.png" width=70%></center>
<center><img src=".\images\part23.png" width=70%></center>
<center><img src=".\images\part24.png" width=70%></center>
<p><strong>第三部分：讲述了应用实现细节，包括机器人学习跑步、机械臂的控制、多智能体平台等，并提供了相关的开源代码。</strong></p>
<center><img src=".\images\part31.png" width=70%></center>
<center><img src=".\images\part32.png" width=70%></center>
<center><img src=".\images\part33.png" width=70%></center>
]]></content:encoded>
    </item>
    <item>
      <title>Hello Vuepress</title>
      <link>https://jiangxj.top/vuepress/vuepress/guide/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/guide/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Hello Vuepress</source>
      <content:encoded><![CDATA[<h2 id="使用vuepress-theme-hope主题"> 使用vuepress-theme-hope主题</h2>
<p>本文使用的是<a href="https://vuepress-theme-hope.github.io/v1/zh/#" target="_blank" rel="noopener noreferrer">V1版本</a>。</p>
<div><pre><code> <span>"devDependencies"</span><span>:</span> <span>{</span>
    <span>"vuepress"</span><span>:</span> <span>"^1.9.7"</span><span>,</span>
    <span>"vuepress-theme-hope"</span><span>:</span> <span>"^1.28.1"</span>
  <span>}</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br></div></div><div><p>配置教程</p>
<p>参见：https://vuepress-theme-hope.github.io/v1/zh/basic/tutorial.html</p>
</div>
<h2 id="环境安装"> 环境安装</h2>
<p>需要安装最新的Node.js长期支持版（<strong>不然会报错</strong>）和yarn</p>
<ul>
<li>node.js长期支持版：<a href="https://nodejs.org/zh-cn/" target="_blank" rel="noopener noreferrer">下载地址</a></li>
</ul>
<p>​       安装保持默认设置。</p>
<p>yarn</p>
<p>安装node.js之后，终端执行：</p>
<div><pre><code>npm i -g yarn  #全局安装
</code></pre>
<div><span>1</span><br></div></div><h2 id="创建模板"> 创建模板</h2>
<p>选择合适的位置，创建一个文件夹，在该文件夹下打开终端，并执行：</p>
<div><pre><code>yarn create vuepress-theme-hope src
</code></pre>
<div><span>1</span><br></div></div><p>修改<code>package.json</code>文件，添加<code>scripts</code>：</p>
<div><pre><code><span>"scripts"</span><span>:</span> <span>{</span>
    <span>"docs:build"</span><span>:</span> <span>"vuepress build src"</span><span>,</span>
    <span>"docs:clean-dev"</span><span>:</span> <span>"vuepress dev src --no-cache"</span><span>,</span>
    <span>"docs:dev"</span><span>:</span> <span>"vuepress dev src"</span><span>,</span>
    <span>"docs:eject-theme"</span><span>:</span> <span>"vuepress eject-hope src"</span>
  <span>}</span><span>,</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br></div></div><div><p>提示</p>
<p>启动开发服务器，在终端输入<code>yarn docs:dev</code>并回车。通过<code>localhost:8081</code>预览</p>
<p>终止服务器，<code>ctrl+C</code></p>
</div>
<h2 id="遇到的问题"> 遇到的问题</h2>
<p>在根目录的readme中设置footer内容时使用双引号，会出现YAML Exception。项目无法正常编译。</p>
<h2 id="math"> math</h2>
<p>在math-theme-hope主题安装中，已经配置mdhance，里面包含tex渲染数学公式。</p>
<div><pre><code><span>mdEnhance</span><span>:</span> <span>{</span>
      <span>sub</span><span>:</span> <span>true</span><span>,</span> 
      <span>sup</span><span>:</span> <span>true</span><span>,</span> <span>// 角标</span>
      <span>footnote</span><span>:</span> <span>true</span><span>,</span> <span>// 脚注</span>
      <span>tex</span><span>:</span> <span>true</span><span>,</span>
      <span>flowchart</span><span>:</span> <span>true</span><span>,</span>
      <span>mermaid</span><span>:</span> <span>true</span><span>,</span>
      <span>demo</span><span>:</span> <span>true</span><span>,</span>
      <span>presentation</span><span>:</span> <span>{</span>
        <span>plugins</span><span>:</span> <span>[</span>
          <span>"highlight"</span><span>,</span>
          <span>"math"</span><span>,</span>
          <span>"search"</span><span>,</span>
          <span>"notes"</span><span>,</span>
          <span>"zoom"</span><span>,</span>
          <span>"anything"</span><span>,</span>
          <span>"audio"</span><span>,</span>
          <span>"chalkboard"</span><span>,</span>
        <span>]</span><span>,</span>
      <span>}</span><span>,</span>
    <span>}</span><span>,</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br></div></div><p>安装 markdown-it-katex 插件</p>
<div><pre><code><span>yarn</span> <span>add</span> markdown-it-katex -D
</code></pre>
<div><span>1</span><br></div></div><p>再设置</p>
<div><pre><code><span>// .vuepress/config.js</span>
module<span>.</span>exports <span>=</span> <span>{</span>
  <span>markdown</span><span>:</span> <span>{</span>
    <span>// 开启代码块行号</span>
    <span>lineNumbers</span><span>:</span> <span>true</span><span>,</span>
    <span>// 选择目录层级</span>
    <span>toc</span><span>:</span> <span>{</span> <span>includeLevel</span><span>:</span> <span>[</span><span>1</span><span>,</span> <span>2</span><span>,</span> <span>3</span><span>]</span> <span>}</span><span>,</span>
    <span>// 加载插件</span>
    <span>extendMarkdown</span><span>:</span> <span>md</span> <span>=></span> <span>{</span>
      <span>// use more markdown-it plugins!</span>
      md<span>.</span><span>use</span><span>(</span><span>require</span><span>(</span><span>'markdown-it-katex'</span><span>)</span><span>)</span>
    <span>}</span>
  <span>}</span><span>,</span>
  <span>// 要使 markdown-it-katex 插件生效，还需要设置在每个 html 的 &lt;head> 部分，加上一些依赖的引用。</span>
  <span>head</span><span>:</span> <span>[</span>
    <span>[</span><span>'link'</span><span>,</span> <span>{</span>
      <span>rel</span><span>:</span> <span>'stylesheet'</span><span>,</span>
      <span>href</span><span>:</span> <span>'https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.7.1/katex.min.css'</span>
    <span>}</span><span>]</span><span>,</span>
    <span>[</span><span>'link'</span><span>,</span> <span>{</span>
      <span>rel</span><span>:</span> <span>"stylesheet"</span><span>,</span>
      <span>href</span><span>:</span> <span>"https://cdnjs.cloudflare.com/ajax/libs/github-markdown-css/2.10.0/github-markdown.min.css"</span>
    <span>}</span><span>]</span><span>,</span>
  <span>]</span>
<span>}</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br><span>22</span><br><span>23</span><br><span>24</span><br><span>25</span><br></div></div><p>效果如下：<br>
<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1901em;vertical-align:-0.345em;"></span><span><span></span><span><span><span><span style="height:0.8451em;"><span style="top:-2.655em;"><span style="height:3em;"></span><span><span><span>n</span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.394em;"><span style="height:3em;"></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.345em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.0622em;vertical-align:-0.2481em;"></span><span><span>n</span><span><span><span><span style="height:0.8141em;"><span style="top:-2.4519em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.2481em;"><span></span></span></span></span></span></span></span></span></span></p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:2.4637em;vertical-align:-0.9819em;"></span><span><span><span style="width:0.5em;"></span><span><span><span><span style="height:1.4819em;"><span style="top:-3.4819em;"><span style="height:3.4819em;"></span><span><span><span><span style="top:0em;"><span>[</span></span><span><span><span style="width:0.5em;"></span><span><span><span><span style="height:1.45em;"><span style="top:-3.61em;"><span style="height:3em;"></span><span><span>cos</span><span style="margin-right:0.1667em;"></span><span>ϕ</span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span>−</span><span style="margin-right:0.1667em;"></span><span>sin</span><span style="margin-right:0.1667em;"></span><span>ϕ</span></span></span></span><span>​</span></span><span><span style="height:0.95em;"><span></span></span></span></span></span><span style="width:0.5em;"></span><span style="width:0.5em;"></span><span><span><span><span style="height:1.45em;"><span style="top:-3.61em;"><span style="height:3em;"></span><span><span>sin</span><span style="margin-right:0.1667em;"></span><span>ϕ</span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span>cos</span><span style="margin-right:0.1667em;"></span><span>ϕ</span></span></span></span><span>​</span></span><span><span style="height:0.95em;"><span></span></span></span></span></span><span style="width:0.5em;"></span></span></span><span style="top:0em;"><span>]</span></span></span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;"><span>[</span></span><span><span><span style="width:0.5em;"></span><span><span><span><span style="height:1.45em;"><span style="top:-3.61em;"><span style="height:3em;"></span><span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span>0</span></span></span></span><span>​</span></span><span><span style="height:0.95em;"><span></span></span></span></span></span><span style="width:0.5em;"></span><span style="width:0.5em;"></span><span><span><span><span style="height:1.45em;"><span style="top:-3.61em;"><span style="height:3em;"></span><span><span>0</span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>2</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:0.95em;"><span></span></span></span></span></span><span style="width:0.5em;"></span></span></span><span style="top:0em;"><span>]</span></span></span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;"><span>[</span></span><span><span><span style="width:0.5em;"></span><span><span><span><span style="height:1.45em;"><span style="top:-3.61em;"><span style="height:3em;"></span><span><span>cos</span><span style="margin-right:0.1667em;"></span><span>ϕ</span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span>sin</span><span style="margin-right:0.1667em;"></span><span>ϕ</span></span></span></span><span>​</span></span><span><span style="height:0.95em;"><span></span></span></span></span></span><span style="width:0.5em;"></span><span style="width:0.5em;"></span><span><span><span><span style="height:1.45em;"><span style="top:-3.61em;"><span style="height:3em;"></span><span><span>−</span><span style="margin-right:0.1667em;"></span><span>sin</span><span style="margin-right:0.1667em;"></span><span>ϕ</span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span>cos</span><span style="margin-right:0.1667em;"></span><span>ϕ</span></span></span></span><span>​</span></span><span><span style="height:0.95em;"><span></span></span></span></span></span><span style="width:0.5em;"></span></span></span><span style="top:0em;"><span>]</span></span></span><span style="margin-right:0.2778em;"></span><span>=</span></span><span><span><span style="top:0em;"><span>[</span></span><span><span><span style="width:0.5em;"></span><span><span><span><span style="height:1.4819em;"><span style="top:-3.61em;"><span style="height:3em;"></span><span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span>cos</span><span><span><span><span style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>2</span></span></span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span>ϕ</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>2</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span>sin</span><span><span><span><span style="height:0.8719em;"><span style="top:-3.1208em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>2</span></span></span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span>ϕ</span></span></span><span style="top:-2.3781em;"><span style="height:3em;"></span><span><span><span style="top:0em;">(</span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>2</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="top:0em;">)</span></span><span style="margin-right:0.1667em;"></span><span>cos</span><span style="margin-right:0.1667em;"></span><span>ϕ</span><span style="margin-right:0.1667em;"></span><span>sin</span><span style="margin-right:0.1667em;"></span><span>ϕ</span></span></span></span><span>​</span></span><span><span style="height:0.9819em;"><span></span></span></span></span></span><span style="width:0.5em;"></span><span style="width:0.5em;"></span><span><span><span><span style="height:1.4819em;"><span style="top:-3.61em;"><span style="height:3em;"></span><span><span><span style="top:0em;">(</span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>2</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="top:0em;">)</span></span><span style="margin-right:0.1667em;"></span><span>cos</span><span style="margin-right:0.1667em;"></span><span>ϕ</span><span style="margin-right:0.1667em;"></span><span>sin</span><span style="margin-right:0.1667em;"></span><span>ϕ</span></span></span><span style="top:-2.3781em;"><span style="height:3em;"></span><span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>2</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span>cos</span><span><span><span><span style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>2</span></span></span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span>ϕ</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span>sin</span><span><span><span><span style="height:0.8719em;"><span style="top:-3.1208em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>2</span></span></span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span>ϕ</span></span></span></span><span>​</span></span><span><span style="height:0.9819em;"><span></span></span></span></span></span><span style="width:0.5em;"></span></span></span><span style="top:0em;"><span>]</span></span></span></span></span></span></span><span>​</span></span><span><span style="height:0.9819em;"><span></span></span></span></span></span><span style="width:0.5em;"></span></span></span></span></span></span></span></p>
<h2 id="git-hub-tables"> git hub tables</h2>
<table>
<thead>
<tr>
<th>Tables</th>
<th style="text-align:center">Are</th>
<th style="text-align:right">Cool</th>
</tr>
</thead>
<tbody>
<tr>
<td>col 3 is</td>
<td style="text-align:center">right-aligned</td>
<td style="text-align:right">$1600</td>
</tr>
<tr>
<td>col 2 is</td>
<td style="text-align:center">centered</td>
<td style="text-align:right">$12</td>
</tr>
<tr>
<td>zebra stripes</td>
<td style="text-align:center">are neat</td>
<td style="text-align:right">$1</td>
</tr>
</tbody>
</table>
<h2 id="container"> container</h2>
<div><p>提示</p>
<p>this is tips</p>
</div>
<div><p>注意</p>
<p>this is warning</p>
</div>
<div><p>STOP</p>
<p>this is danger warning</p>
</div>
<details><summary>custom title</summary>
<p>more details here</p>
</details>
<h2 id="语法高亮"> 语法高亮</h2>
<p>代码块内某些行高亮。{}里面行号之间不能有空格。</p>
<div><div><br><div>&nbsp;</div><br><br><br><div>&nbsp;</div><br><br></div><pre><code><span><span>#</span><span>version</span> <span><span>330</span> core</span></span>
<span>(</span><span>layout</span> location <span>=</span> <span>0</span><span>)</span> <span>in</span> <span>vec3</span> a_Pos

<span>void</span> <span>main</span><span>(</span><span>)</span>
<span>{</span>
    gl_Position <span>=</span> a_Pos<span>;</span>
<span>}</span>
</code></pre><div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br></div></div><h2 id="部署"> 部署</h2>
<p>按文档来，没成功。git push -f xxxxx master 会在github库中新建一个master分支，而且有时候推送失败，原因不明。<br>
因为我安装了 wsl2，执行bash命令会进入Ubantu18.04，也导致shell脚本出错。<br>
最终，解决方法如下：</p>
<p>在package.json中添加npm命令，用于执行shell脚本。</p>
<div><div><br><br><br><div>&nbsp;</div><br><br></div><pre><code><span>"scripts"</span><span>:</span> <span>{</span>
    <span>"docs:dev"</span><span>:</span> <span>"vuepress dev docs"</span><span>,</span>
    <span>"docs:build"</span><span>:</span> <span>"vuepress build docs"</span><span>,</span>
    <span>"docs:deploy"</span><span>:</span> <span>"deploy.sh"</span>
  <span>}</span><span>,</span>
</code></pre><div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br></div></div><p>手动clone GithubPage关联库 https://github.com/username/username.github.io.git 到本地，并放在目录project/docs/.vuepress/下。(和生成的dist目录同级)</p>
<p>deploy.sh内容为</p>
<div><pre><code><span>#!/usr/bin/env sh</span>

<span># abort on errors</span>
<span>set</span> -e
<span>#生成预览</span>
<span>yarn</span> docs:dev
<span># build</span>
<span>yarn</span> docs:build

<span># navigate into the build output directory</span>
<span>cd</span> src/.vuepress/dist

<span># 将一个目录下的一些文件移动到另一个目录下</span>
<span>cp</span> -R * <span>..</span>/username.github.io

<span>cd</span> <span>..</span>/username.github.io

<span>git</span> <span>add</span> -A
<span>git</span> commit -m <span>'deploy'</span>
<span>git</span> push

<span>cd</span> -

<span># 执行完不立即退出shell，便于查看错误信息。</span>
<span>read</span> -n <span>1</span> -p <span>'Press any key to continue...'</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br><span>19</span><br><span>20</span><br><span>21</span><br><span>22</span><br><span>23</span><br><span>24</span><br><span>25</span><br></div></div><p>deploy.sh 实际上完成了三件事</p>
<ol>
<li>执行<code>yarn docs:build</code>生成要发布的内容。(重复生成时 dist 文件夹会先被清空，再生成新文件。)</li>
<li>将 <code>dist</code> 文件夹中的所有文件都复制到 username.githbu.io 文件夹中</li>
<li>在 username.githbu.io 文件夹中(GithubPage关联的库)，提交修改并推送到远程库</li>
</ol>
<h2 id="部署后不能渲染"> 部署后不能渲染</h2>
<p>在<code>config.js</code>中添加GitHub的仓库名：</p>
<div><pre><code>module<span>.</span>exports <span>=</span> <span>config</span><span>(</span><span>{</span>

  <span>dest</span><span>:</span> <span>"./dist"</span><span>,</span> <span>//输出目录</span>
  <span>base</span><span>:</span> <span>'/vuepress/'</span><span>,</span> <span>//仓库名</span>
  <span>head</span><span>:</span> <span>[</span>
 <span>...</span><span>.</span>
 <span>...</span><span>.</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br></div></div><h2 id="全局computed"> 全局Computed</h2>
<p>因为 ketex 公式分隔符是 'dollar', 和 vue 提供的全局变量前缀冲突了。所以一页只能有一个全局变量保持正常。</p>
]]></content:encoded>
    </item>
    <item>
      <title>深度强化学习（Deep Reinforcement Learning）基础1</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/DRL/%E6%B7%B1%E5%BA%A6%E5%BC%BA%E5%8C%96%E5%AD%A6%E4%B9%A0%E5%9F%BA%E7%A1%801/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/DRL/%E6%B7%B1%E5%BA%A6%E5%BC%BA%E5%8C%96%E5%AD%A6%E4%B9%A0%E5%9F%BA%E7%A1%801/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">深度强化学习（Deep Reinforcement Learning）基础1</source>
      <description>深度强化学习——深度学习部分简介。</description>
      <category>深度强化学习</category>
      <pubDate>Thu, 03 Mar 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<blockquote>
<p>关于深度学习推荐两本书：</p>
<p>Pattern Recognition and Machine Learning（Bishop, 2006）</p>
<p>Deep Learning（Goodfellow et al., 2016）</p>
</blockquote>
<h2 id="判别模型和生成模型"> 判别模型和生成模型</h2>
<p>​		深度学习模型大致分为两类：<strong>判别模型（Discriminative Models）<strong>和</strong>生成模型（Generative Models）</strong>。</p>
<p>​		判别模型用于<strong>建模条件概率</strong><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>p</span><span>(</span><span><span><span><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span><span style="margin-right:0.03588em;">y</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.03588em;">y</span></span><span>∣</span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>x</span></span><span>)</span></span></span></span>，其中<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>x</span></span></span></span></span>代表输入数据，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span><span><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span><span style="margin-right:0.03588em;">y</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.03588em;">y</span></span></span></span></span>是输出目标。判别模型基于输入，预测相应的标签<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span><span><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span><span style="margin-right:0.03588em;">y</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.03588em;">y</span></span></span></span></span>。</p>
<p>判别模型大多应用于需要判断的任务，如分类任务，回归任务。</p>
<p>​		分类任务中，模型需要根据输入数据从备选类别中选择正确的目标类别，典型包括二分类、多标签分类等。</p>
<p>​		很多情况下，<em>一个分类模型并不直接指定目标类别，而是给每一个备选类别计算一个概率</em>，这主要是为了便于在训练阶段对模型进行优化。深度学习在<strong>图像分类、文本分类</strong>任务上取得很大成功。</p>
<p>​		<strong>分类任务的输出均为离散的类别标签</strong>。</p>
<p>​		<strong>回归任务的输出是连续的数值</strong>，如利用过去的交通数据预测未来一段时间内的车速<sup></sup><sup></sup>。</p>
<p>​		生成模型用于<strong>建模联合概率</strong><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>p</span><span>(</span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>x</span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span><span><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span><span style="margin-right:0.03588em;">y</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.03588em;">y</span></span><span>)</span></span></span></span>。生成模型通常对可观测数据的分布进行建模，达到生成可观测数据的目的。比如生成对抗网络（Generative Adversarial Networks, GANs）（用于生成图像、重构图像和对图像去噪）<sup></sup>。</p>
<p>​		DL技术和可观测数据的分布没有显式关系，<strong>DL技术更关注生成的样本和可观测的真实样本之间的相似程度</strong>。朴素贝叶斯（Naive Bayes）<sup></sup>的生成模型也用于分类任务<sup></sup>。</p>
<p>​      判别模型关注的是哪一个标签更适合可观测数据，生成模型尝试建模可观察数据的分布。</p>
<p>举例说明：</p>
<p>​	1. 朴素贝叶斯对似然概率（Likelihood）<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>p</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>∣</span><span>x</span><span>)</span></span></span></span>建模，即可观测数据在给定标签情况下的条件概率。（判别模型）</p>
<p>​	2. 当学习了联合概率分布<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>p</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span></span></span></span>之后即可学会判别，给定观测输入<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span>，输出目标为1的概率为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>p</span><span>(</span><span style="margin-right:0.03588em;">y</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>1∣</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.53em;vertical-align:-0.52em;"></span><span><span></span><span><span><span><span style="height:1.01em;"><span style="top:-2.655em;"><span style="height:3em;"></span><span><span><span>p</span><span>(</span><span>x</span><span>)</span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.485em;"><span style="height:3em;"></span><span><span><span>p</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.03588em;">y</span><span>=</span><span>1</span><span>)</span></span></span></span></span><span>​</span></span><span><span style="height:0.52em;"><span></span></span></span></span></span><span></span></span></span></span></span>。（生成模型）</p>
<p>​		<em><strong>大多数深度神经网络是判别模型</strong>，很多生成类任务在具体实现中都可以简化为分类或者回归问题。</em></p>
<h2 id="感知器"> 感知器</h2>
<p><font size=5> <strong>单输出</strong> </font></p>
<p><strong>感知器</strong>（Perceptron）是一个单一层网络结构。</p>
<center><img src="./images/感知器.png" title='单输出感知器'></center>
<p>其中：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.02691em;">w</span></span></span></span>是相对应的网络权重，是决定影响大小的因素，若没有影响，则<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.02691em;">w</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span>。</p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span>b</span></span></span></span>为<strong>偏差（bias）</strong>，用于偏移神经网络的输出。偏差可以帮助神经网络更好的学习数据。</p>
<p><font size=5> <strong>决策边界</strong> </font></p>
<center><img src="./images/决策边界.png" width=80% title='线性模型决策边界'></center>
<p>在<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span>b</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span>，没有偏差条件下，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.04398em;">z</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7333em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>决策边界必须穿过坐标系的原点，但不符合数据的分布（上左图），因而需要选择好权重和偏差值。</p>
<p>当有三个输入的时候，决策边界会是一个平面，称为<strong>超平面（Hyperplane</strong>）。</p>
<p><font size=5> <strong>多输出</strong> </font></p>
<center><img src="./images/多输出感知器.png" title='多输出感知器'></center>
<p>输出层称为全连接层（Fully-Connected(FC) Layer），全连接层可以被矩阵乘法实现：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.04398em;">z</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7667em;vertical-align:-0.0833em;"></span><span><span><span><span><span style="height:0.6833em;"></span><span><span><span style="margin-right:0.13889em;">W</span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.13889em;">W</span><span>x</span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6944em;"></span><span><span><span><span><span style="height:0.6944em;"></span><span><span><span>b</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>b</span></span></span></span></span></span></p>
<p>其中，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7224em;vertical-align:-0.0391em;"></span><span><span><span><span><span style="height:0.6833em;"></span><span><span><span style="margin-right:0.13889em;">W</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.13889em;">W</span></span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7713em;"></span><span><span>R</span><span><span><span><span style="height:0.7713em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>m</span><span>×</span><span>n</span></span></span></span></span></span></span></span></span></span></span></span>表示权重矩阵，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span style="margin-right:0.04398em;">z</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6889em;"></span><span><span>R</span><span><span><span><span style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>m</span></span></span></span></span></span></span></span></span></span></span></span>，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>x</span></span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6889em;"></span><span><span>R</span><span><span><span><span style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>n</span></span></span></span></span></span></span></span></span></span></span></span>，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7335em;vertical-align:-0.0391em;"></span><span><span><span><span><span style="height:0.6944em;"></span><span><span><span>b</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>b</span></span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6889em;"></span><span><span>R</span><span><span><span><span style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>m</span></span></span></span></span></span></span></span></span></span></span></span>分别表示输出、输入和偏差向量。</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:4.26em;vertical-align:-1.88em;"></span><span><span><span><span><span><span style="height:2.35em;"><span style="top:-1.95em;"><span style="height:3.155em;"></span><span><span>⎣</span></span></span><span style="top:-3.097em;"><span style="height:3.155em;"></span><span style="height:0.616em;width:0.6667em;"><svg xmlns="http://www.w3.org/2000/svg" width='0.6667em' height='0.616em' style='width:0.6667em' viewBox='0 0 666.67 616' preserveAspectRatio='xMinYMin'><path d='M319 0 H403 V616 H319z M319 0 H403 V616 H319z'/></svg></span></span><span style="top:-4.35em;"><span style="height:3.155em;"></span><span><span>⎡</span></span></span></span><span>​</span></span><span><span style="height:1.85em;"><span></span></span></span></span></span></span><span><span><span><span><span><span style="height:2.38em;"><span style="top:-5.04em;"><span style="height:3.5em;"></span><span><span><span>a</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>11</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-3.18em;"><span style="height:3.5em;"></span><span></span></span><span style="top:-1.98em;"><span style="height:3.5em;"></span><span><span>0</span></span></span></span><span>​</span></span><span><span style="height:1.88em;"><span></span></span></span></span></span><span style="width:0.5em;"></span><span style="width:0.5em;"></span><span><span><span><span style="height:2.38em;"><span style="top:-5.04em;"><span style="height:3.5em;"></span><span><span>…</span></span></span><span style="top:-3.18em;"><span style="height:3.5em;"></span><span><span>⋱</span></span></span><span style="top:-1.98em;"><span style="height:3.5em;"></span><span></span></span></span><span>​</span></span><span><span style="height:1.88em;"><span></span></span></span></span></span><span style="width:0.5em;"></span><span style="width:0.5em;"></span><span><span><span><span style="height:2.38em;"><span style="top:-5.2275em;"><span style="height:3.6875em;"></span><span><span><span>a</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>1</span><span>n</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-3.3675em;"><span style="height:3.6875em;"></span><span><span><span>⋮</span><span style="border-right-width:0em;border-top-width:1.5em;bottom:0em;"></span></span></span></span><span style="top:-2.1675em;"><span style="height:3.6875em;"></span><span><span><span>a</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>nn</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:1.88em;"><span></span></span></span></span></span></span></span><span><span><span><span><span style="height:2.35em;"><span style="top:-1.95em;"><span style="height:3.155em;"></span><span><span>⎦</span></span></span><span style="top:-3.097em;"><span style="height:3.155em;"></span><span style="height:0.616em;width:0.6667em;"><svg xmlns="http://www.w3.org/2000/svg" width='0.6667em' height='0.616em' style='width:0.6667em' viewBox='0 0 666.67 616' preserveAspectRatio='xMinYMin'><path d='M263 0 H347 V616 H263z M263 0 H347 V616 H263z'/></svg></span></span><span style="top:-4.35em;"><span style="height:3.155em;"></span><span><span>⎤</span></span></span></span><span>​</span></span><span><span style="height:1.85em;"><span></span></span></span></span></span></span></span></span></span></span></span></p>
<h2 id="参考文献"> 参考文献</h2>
<hr>
<section>
<ol>
<li id="footnote1"><p>LIAO B, ZHANG J, WU C, et al., 2018b.Deep sequence learning with auxiliary information for traffic prediction[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery &amp; Data Mining. ACM: 537-546. </p>
</li>
<li id="footnote2"><p>LIAO B, ZHANG J, CAI M, et al., 2018a. Dest-ResNet: A deep spatiotemporal residual network for hotspot traffic speed prediction[C]//2018 ACM Multimedia Conference on Multimedia Conference. ACM: 1883-1891. </p>
</li>
<li id="footnote3"><p>GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al., 2014. Generative Adversarial Nets[C]. Proceedings of the Neural Information Processing Systems (Advances in Neural Information Processing Systems) Conference. </p>
</li>
<li id="footnote4"><p>NG A Y, JORDAN M I, 2002. On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes[C]//Proceedings of the Neural Information Processing Systems (Advances in Neural Information Processing Systems) Conference. 841-848. </p>
</li>
<li id="footnote5"><p>RISH I, et al., 2001.An empirical study of the naive bayes classifier[C]//International Joint Conference on Artificial Intelligence 2001 workshop on empirical methods in artificial intelligence: volume 3. 41-46 </p>
</li>
</ol>
</section>
]]></content:encoded>
    </item>
    <item>
      <title>Custom Layout</title>
      <link>https://jiangxj.top/vuepress/vuepress/layout/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/layout/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Custom Layout</source>
      <content:encoded><![CDATA[<p>You can use slots with markdown and component support to custom page layout.</p>
<div><p>注意</p>
<p>This is just a demo, you should add styles according to your own needs.</p>


</div>
<template #page-top><p>Page top content</p>
</template><template #page-bottom><p>Page bottom content</p>
</template><template #content-top><p>Content top content</p>
</template><template #content-bottom><p>Content bottom content</p>
</template><template #navbar-start><p>Navbar start content</p>
</template><template #navbar-center><p>Navbar center content</p>
</template><template #navbar-end><p>Navbar end content</p>
</template><template #sidebar-top><p>Sidebar top content</p>
</template><template #sidebar-center><p>Sidebar center content</p>
</template><template #sidebar-bottom><p>Sidebar bottom content</p>
</template><p>For details, see <a href="https://vuepress-theme-hope.github.io/guide/layout/custom/" target="_blank" rel="noopener noreferrer">Custom layout</a>.</p>
]]></content:encoded>
    </item>
    <item>
      <title>决策树</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/MachineLearning/DecisionTree/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/MachineLearning/DecisionTree/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">决策树</source>
      <description>决策树方算法原理及实现</description>
      <category>MachineLearning</category>
      <pubDate>Fri, 02 Dec 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="概述"> 概述</h2>
<p>决策树（Decision Tree）学习是应用最广的<strong>归纳推理</strong>（Inductive Reasoning，由个别到一般的推理）算法之一。是一种逼近离散函数的方法，且对噪声数据有很好的鲁棒性，能够学习析取表达式。</p>
]]></content:encoded>
    </item>
    <item>
      <title>分类与回归</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/MachineLearning/LinearModel/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/MachineLearning/LinearModel/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">分类与回归</source>
      <description>机器学习中的分类与回归任务</description>
      <category>MachineLearning</category>
      <pubDate>Fri, 02 Dec 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="概述"> 概述</h2>
<h3 id="回归与分类的区别"> 回归与分类的区别</h3>
<ol>
<li>
<p>分类输出的数据类型是离散数据；回归输出的连续数据。</p>
</li>
<li>
<p>分类得到的是一个决策面，用于对数据集的数据进行分类；回归得到的是一条最优拟合线，较好地拟合数据集各个点。</p>
</li>
<li>
<p>分类模型一般评价指标是准确率；回归模型一般用决定系数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8141em;"></span><span><span style="margin-right:0.00773em;">R</span><span><span><span><span style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span>来评估，表示占比多少的数据点被回归线拟合。</p>
</li>
</ol>
<p>机器学习中常见的是分类任务。</p>
<p>分类任务分为<strong>线性</strong>（linear）和<strong>非线性</strong>（nonlinear）。</p>
<h3 id="线性模型"> 线性模型</h3>
<p>线性模型使用线性的函数表达式对样本进行分类，即划分边界是一个超平面。二维空间是一条直线，三维空间是一个平面。</p>
<p>给定一个<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.02778em;">D</span></span></span></span>维样本<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>x</span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.0913em;vertical-align:-0.25em;"></span><span>[</span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.02778em;">D</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>]</span><span><span><span><span style="height:0.8413em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span></span></span></span>，其线性组合函数为：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:3.0513em;vertical-align:-1.2757em;"></span><span><span><span><span><span><span style="height:1.7757em;"><span style="top:-3.9357em;"><span style="height:3em;"></span><span><span style="margin-right:0.10764em;">f</span><span>(</span><span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>x</span></span><span>;</span><span style="margin-right:0.1667em;"></span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span style="margin-right:0.02691em;">w</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.02691em;">w</span></span></span><span>)</span></span></span><span style="top:-2.3843em;"><span style="height:3em;"></span><span></span></span></span><span>​</span></span><span><span style="height:1.2757em;"><span></span></span></span></span></span><span><span><span><span style="height:1.7757em;"><span style="top:-3.9357em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span>⋯</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.02778em;">D</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>x</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.02778em;">D</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span>b</span></span></span><span style="top:-2.3843em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span style="margin-right:0.02691em;">w</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.02691em;">w</span></span><span><span><span><span style="height:0.8913em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>x</span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span>b</span></span></span></span><span>​</span></span><span><span style="height:1.2757em;"><span></span></span></span></span></span></span></span></span></span></span></span></p>
<p>其中，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span style="margin-right:0.02691em;">w</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.02691em;">w</span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.0913em;vertical-align:-0.25em;"></span><span>[</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.02778em;">D</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>]</span><span><span><span><span style="height:0.8413em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span></span></span></span>是<strong>权重</strong>（weight）向量，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span>b</span></span></span></span>是<strong>偏置</strong>（bias）。</p>
<p>输出目标函数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span style="margin-right:0.03588em;">y</span></span></span></span>是离散的标签值，而<strong>判别函数</strong>（Discriminant Function）<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10764em;">f</span><span>(</span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>x</span></span><span>;</span><span style="margin-right:0.1667em;"></span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span style="margin-right:0.02691em;">w</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.02691em;">w</span></span><span>)</span></span></span></span>的输出值是实数，之间需要有一个非线性的<strong>决策函数</strong>（Decision Function）<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.03588em;">g</span><span>(</span><span>⋅</span><span style="margin-right:0.1667em;"></span><span>)</span></span></span></span>来预测输出目标。</p>
<blockquote>
<p>神经网络中，决策函数也称为<strong>激活函数</strong>（activation function）。</p>
</blockquote>
<p>在二分类问题中，以<strong>符号函数</strong>（Sign Function）作为决策函数，定义如下：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:4.8em;vertical-align:-2.15em;"></span><span><span><span><span><span><span style="height:2.65em;"><span style="top:-5.56em;"><span style="height:3.75em;"></span><span><span style="margin-right:0.03588em;">g</span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;">(</span><span style="margin-right:0.10764em;">f</span><span>(</span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>x</span></span><span>;</span><span style="margin-right:0.1667em;"></span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span style="margin-right:0.02691em;">w</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.02691em;">w</span></span><span>)</span><span style="top:0em;">)</span></span></span></span><span style="top:-3.15em;"><span style="height:3.75em;"></span><span></span></span></span><span>​</span></span><span><span style="height:2.15em;"><span></span></span></span></span></span><span><span><span><span style="height:2.65em;"><span style="top:-5.56em;"><span style="height:3.75em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span>sgn</span></span></span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;">(</span><span style="margin-right:0.10764em;">f</span><span>(</span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>x</span></span><span>;</span><span style="margin-right:0.1667em;"></span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span style="margin-right:0.02691em;">w</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.02691em;">w</span></span><span>)</span><span style="top:0em;">)</span></span></span></span><span style="top:-3.15em;"><span style="height:3.75em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>≜</span><span style="margin-right:0.2778em;"></span><span><span style="top:0em;"><span>{</span></span><span><span><span><span><span><span style="height:1.69em;"><span style="top:-3.69em;"><span style="height:3.008em;"></span><span><span>+</span><span>1</span><span style="margin-right:1em;"></span><span>i</span><span style="margin-right:0.10764em;">f</span><span style="margin-right:1em;"></span><span style="margin-right:0.10764em;">f</span><span>(</span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>x</span></span><span>;</span><span style="margin-right:0.1667em;"></span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span style="margin-right:0.02691em;">w</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.02691em;">w</span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>&gt;</span><span style="margin-right:0.2778em;"></span><span>0</span><span>,</span></span></span><span style="top:-2.25em;"><span style="height:3.008em;"></span><span><span>−</span><span>1</span><span style="margin-right:1em;"></span><span>i</span><span style="margin-right:0.10764em;">f</span><span style="margin-right:1em;"></span><span style="margin-right:0.10764em;">f</span><span>(</span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>x</span></span><span>;</span><span style="margin-right:0.1667em;"></span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span style="margin-right:0.02691em;">w</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.02691em;">w</span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span><span>0.</span></span></span></span><span>​</span></span><span><span style="height:1.19em;"><span></span></span></span></span></span></span></span><span></span></span></span></span></span><span>​</span></span><span><span style="height:2.15em;"><span></span></span></span></span></span></span></span></span></span></span></span></p>
<p>其中<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10764em;">f</span><span>(</span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span>x</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span>x</span></span><span>;</span><span style="margin-right:0.1667em;"></span><span><span><span><span><span style="height:0.4306em;"></span><span><span><span style="margin-right:0.02691em;">w</span></span></span><span></span></span></span></span><span style="margin-right:0.0502em;"></span><span style="margin-right:0.02691em;">w</span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span>不预测。</p>
]]></content:encoded>
    </item>
    <item>
      <title>主成分分析（PCA）</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/MachineLearning/PCA/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/MachineLearning/PCA/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">主成分分析（PCA）</source>
      <description>关于主成分分析（PCA）的概念、原理、实现代码的简介。</description>
      <category>MachineLearning</category>
      <pubDate>Tue, 01 Mar 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<blockquote>
<p>参考：李航《统计学习方法》（第2版）</p>
</blockquote>
<h2 id="概念"> 概念</h2>
<p>主成分分析（principal component analysis, PCA）是一种无监督学习方法（不需要先验知识）。</p>
<p>PCA利用正交变换把由线性相关变量表示的观测数据转化为少数几个由线性无关变量表示的数据，线性无关的变量称为<strong>主成分</strong>。</p>
<p>其中，主成分的个数通常小于原始变量的个数，所以PCA属于<strong>降维</strong>方法。</p>
<p>PCA主要用于发现数据中的基本结构，即<strong>数据中变量之间的关系</strong>。</p>
<h2 id="基本思想"> 基本思想</h2>
<ul>
<li>数据集合中的样本由实数空间（正交坐标系） 中的点表示，空间的一个坐标轴表示一个变量，规范化处理后得到的数据分布在原点附近。</li>
<li>对原坐标系中的数据进行主成分分析等价于进行坐标系旋转变换，将数据投影到新坐标系的坐标轴上</li>
<li>新坐标系的第一坐标轴、第二坐标轴等分别表示第一主成分、 第二主成分等</li>
<li>数据在每一轴上的坐标值的平方表示相应变量的方差</li>
<li>这个坐标系是在所有可能的新的坐标系中，坐标轴上的方差的和最大的</li>
</ul>
<p>主成分分析对数据进行<strong>正交变换</strong>，对原坐标系进行旋转变换，并将数据在新坐标系中表示：</p>
<center><img src="./PCA/正交变换.png" title="正交变换" width=60% ></center>
<p>由随机变量性质可知：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:4.654em;vertical-align:-2.077em;"></span><span><span><span><span><span><span style="height:2.577em;"><span style="top:-4.577em;"><span style="height:2.8913em;"></span><span></span></span><span style="top:-3.0257em;"><span style="height:2.8913em;"></span><span></span></span><span style="top:-1.4743em;"><span style="height:2.8913em;"></span><span></span></span></span><span>​</span></span><span><span style="height:2.077em;"><span></span></span></span></span></span><span><span><span><span style="height:2.577em;"><span style="top:-4.6857em;"><span style="height:3em;"></span><span><span></span><span><span><span>E</span></span></span><span>(</span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span style="margin-right:0.0037em;">α</span><span><span><span><span style="height:0.8913em;"><span style="top:-2.453em;margin-left:-0.0037em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span><span>​</span></span><span><span style="height:0.247em;"><span></span></span></span></span></span></span><span>μ</span><span>,</span><span> </span><span style="margin-right:0.1667em;"></span><span>i</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span>m</span></span></span><span style="top:-3.1343em;"><span style="height:3em;"></span><span><span></span><span><span><span>var</span></span></span><span>(</span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span style="margin-right:0.0037em;">α</span><span><span><span><span style="height:0.8913em;"><span style="top:-2.453em;margin-left:-0.0037em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span><span>​</span></span><span><span style="height:0.247em;"><span></span></span></span></span></span></span><span>Σ</span><span><span style="margin-right:0.0037em;">α</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0037em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span> </span><span style="margin-right:0.1667em;"></span><span>i</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span>m</span></span></span><span style="top:-1.583em;"><span style="height:3em;"></span><span><span></span><span><span><span style="margin-right:0.01389em;">cov</span></span></span><span>(</span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span style="margin-right:0.0037em;">α</span><span><span><span><span style="height:0.8913em;"><span style="top:-2.453em;margin-left:-0.0037em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span><span>​</span></span><span><span style="height:0.247em;"><span></span></span></span></span></span></span><span>Σ</span><span><span style="margin-right:0.0037em;">α</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0037em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>,</span><span> </span><span style="margin-right:0.1667em;"></span><span>i</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span>m</span><span>;</span><span> </span><span> </span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.05724em;">j</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span>m</span></span></span></span><span>​</span></span><span><span style="height:2.077em;"><span></span></span></span></span></span></span></span></span></span></span></span></p>
]]></content:encoded>
    </item>
    <item>
      <title>机器学习的基本设计方法和途径</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/MachineLearning/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E7%9A%84%E5%9F%BA%E6%9C%AC%E8%AE%BE%E8%AE%A1%E6%96%B9%E6%B3%95%E5%92%8C%E9%80%94%E5%BE%84/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/MachineLearning/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E7%9A%84%E5%9F%BA%E6%9C%AC%E8%AE%BE%E8%AE%A1%E6%96%B9%E6%B3%95%E5%92%8C%E9%80%94%E5%BE%84/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">机器学习的基本设计方法和途径</source>
      <description>CMU-Mitchell版机器学习关于-学习-问题的描述。</description>
      <category>MachineLearning</category>
      <pubDate>Fri, 02 Dec 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="学习问题的描述"> 学习问题的描述</h2>
<p><strong>Definition</strong>: A computer program is said to <strong>learn</strong> from experience <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05764em;">E</span></span></span></span> with respect to some class of tasks <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">T</span></span></span></span> and performance measure <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">P</span></span></span></span>, if its performance at tasks in <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">T</span></span></span></span>, as measured by <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">P</span></span></span></span>, improves with experience <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05764em;">E</span></span></span></span>.</p>
<p>利用经验自我提高的计算机程序。</p>
<h2 id="设计一个学习系统步骤"> 设计一个学习系统步骤</h2>
<p>1、选择训练方式</p>
<p>选取训练经验的类型，使系统从中学习。训练经验属性包括：</p>
<ul>
<li>能否为系统的决策提供<strong>直接</strong>或<strong>间接</strong>的反馈。间接信息需要学习系统推断，需要考虑每一步的学习对最终结果的贡献程度。</li>
<li>学习器可以在多大程度上控制训练样本序列。包括监督学习，非监督学习，半监督学习等。</li>
<li>训练样例能多好的表示实例分布。目前的ML理论大多基于训练样本和测试样本分布一致性的前提，这样通过指标<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">P</span></span></span></span>来判断学习的可信度。可能会造成过拟合或欠拟合问题。</li>
</ul>
<p>2、选择目标函数</p>
<p>最优化问题（optimization problem）大多可以理解为在某个先验已知的巨大搜索空间，但<strong>最佳的搜索策略</strong>未知。把对于任务<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">T</span></span></span></span>提高性能<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">P</span></span></span></span>的问题可以简化为学习目标函数（target function）最优问题。</p>
<p>事实上，学习算法很难完美的学习到一个理想的目标函数，仅得到目标函数的某个<strong>近似</strong>（approximation），称为函数逼近（function approximation）。</p>
<p>3、选择目标函数逼近算法</p>
<p>对于一个状态<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span>b</span></span></span></span>和其训练值（i.e. 准确值）<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">V</span><span>(</span><span>b</span><span>)</span></span></span></span>，需要定义目标函数输出值<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1968em;vertical-align:-0.25em;"></span><span><span><span><span style="height:0.9468em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.22222em;">V</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.25em;"><span>^</span></span></span></span></span></span></span><span>(</span><span>b</span><span>)</span></span></span></span>对训练值的拟合程度，找到最佳拟合（best fit）。一个方式是求解<strong>最小化误差平方</strong>：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05764em;">E</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.004em;vertical-align:-0.65em;"></span><span style="position:relative;top:0em;">∑</span><span style="margin-right:0.1667em;"></span><span><span><span style="top:0em;"><span>(</span></span><span style="margin-right:0.22222em;">V</span><span>(</span><span>b</span><span>)</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span><span><span><span style="height:0.9468em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.22222em;">V</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.25em;"><span>^</span></span></span></span></span></span></span><span>(</span><span>b</span><span>)</span><span style="top:0em;"><span>)</span></span></span><span><span><span><span style="height:1.354em;"><span style="top:-3.6029em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></span></p>
<p>目标是寻找权重（i.e. 寻找<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.9468em;"></span><span><span><span><span style="height:0.9468em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.22222em;">V</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.25em;"><span>^</span></span></span></span></span></span></span></span></span></span>），使得<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05764em;">E</span></span></span></span>最小，即寻找给定观测训练数据下的<strong>最可能假设</strong>（hypothesis）。</p>
<h2 id="lms方法"> LMS方法</h2>
<p>最小均方法（Least Mean Squares， LMS）</p>
<p>对于每一个训练样本<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>b</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">V</span><span>(</span><span>b</span><span>)</span><span style="margin-right:0.2778em;"></span><span>&gt;</span></span></span></span>：</p>
<ul>
<li>使用当前的权重计算<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1968em;vertical-align:-0.25em;"></span><span><span><span><span style="height:0.9468em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.22222em;">V</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.25em;"><span>^</span></span></span></span></span></span></span><span>(</span><span>b</span><span>)</span></span></span></span></li>
<li>对于每一个权重<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>，进行如下更新：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>:=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7333em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.8em;vertical-align:-0.65em;"></span><span style="margin-right:0.03588em;">η</span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;"><span>(</span></span><span style="margin-right:0.22222em;">V</span><span>(</span><span>b</span><span>)</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span><span><span><span style="height:0.9468em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.22222em;">V</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.25em;"><span>^</span></span></span></span></span></span></span><span>(</span><span>b</span><span>)</span><span style="top:0em;"><span>)</span></span></span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span></li>
</ul>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span style="margin-right:0.03588em;">η</span></span></span></span>（i.e. <strong>学习率</strong>）用于调整权重值更新的幅度。</p>
<table>
<thead>
<tr>
<th>例子</th>
<th>说明</th>
</tr>
</thead>
<tbody>
<tr>
<td>当<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.8em;vertical-align:-0.65em;"></span><span><span style="top:0em;"><span>(</span></span><span style="margin-right:0.22222em;">V</span><span>(</span><span>b</span><span>)</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span><span><span><span style="height:0.9468em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.22222em;">V</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.25em;"><span>^</span></span></span></span></span></span></span><span>(</span><span>b</span><span>)</span><span style="top:0em;"><span>)</span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span></td>
<td>最优拟合，权值不更新</td>
</tr>
<tr>
<td>当<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.8em;vertical-align:-0.65em;"></span><span><span style="top:0em;"><span>(</span></span><span style="margin-right:0.22222em;">V</span><span>(</span><span>b</span><span>)</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span><span><span><span style="height:0.9468em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.22222em;">V</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.25em;"><span>^</span></span></span></span></span></span></span><span>(</span><span>b</span><span>)</span><span style="top:0em;"><span>)</span></span></span><span style="margin-right:0.2778em;"></span><span>&gt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span></td>
<td><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1968em;vertical-align:-0.25em;"></span><span><span><span><span style="height:0.9468em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.22222em;">V</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.25em;"><span>^</span></span></span></span></span></span></span><span>(</span><span>b</span><span>)</span></span></span></span>太低，权值会根据数据对应的特征值增加一定比例，调高权值来降低和训练值的误差。</td>
</tr>
<tr>
<td>当参数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span></td>
<td>该样本特征值权重不会被更新，样本可省去</td>
</tr>
</tbody>
</table>
<p>LMS相当于是一个<strong>泛化器</strong>（Generalizer），以训练样本作为输入，输出一个用学习到的权重值<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>描述的函数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.9468em;"></span><span><span><span><span style="height:0.9468em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.22222em;">V</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.25em;"><span>^</span></span></span></span></span></span></span></span></span></span>，这是对目标函数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.22222em;">V</span></span></span></span>的一个近似估计，从特定的训练样本中泛化而来，使得能够尽可能多的覆盖训练样本及样本之外的情形。</p>
<h2 id="references"> References</h2>
<p>零基础入门深度学习-感知器：https://www.zybuluo.com/hanbingtao/note/433855</p>
]]></content:encoded>
    </item>
    <item>
      <title>机器学习算法理解</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/MachineLearning/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/MachineLearning/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">机器学习算法理解</source>
      <description>整理的关于机器学习基础算法的原理、实现等内容。</description>
      <content:encoded><![CDATA[<p>PCA</p>
<p>其它待更新。。。</p>
]]></content:encoded>
    </item>
    <item>
      <title>卷积神经网络理解</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/MachineLearning/%E5%8D%B7%E7%A7%AF%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/MachineLearning/%E5%8D%B7%E7%A7%AF%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">卷积神经网络理解</source>
      <description>理解Convolutional Neural Networks.</description>
      <category>MachineLearning</category>
      <pubDate>Tue, 07 Jun 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="发展历程"> 发展历程</h2>
<p>1998年 Yann LeCun教授在论文Gradient-Based Learning Applied to Document Recognition中提出了LeNet-5，将Backpropagation算法应用到神经网络结构训练中，形成了卷积神经网络（CNN）的雏形。</p>
<p>2012年，在ImageNet图像识别大赛中，Hinton组的论文ImageNet Classification with Deep Convolutional Neural Networks中提到了AlexNet，引入了全新的深层结构和Dropout正则化方法，CNN开始蓬勃发展。</p>
<h2 id="基本概念"> 基本概念</h2>
<p>卷积核（滤波器）：filter。如3×3、5×5之类的。</p>
<p>特征图：feature map。输入和卷积核对应相乘累加计算得到的结果所组成，<strong>一个卷积核对应一个feature map</strong>。</p>
<p>卷积层：Convolutional layer。用于提取底层特征</p>
<p>池化层：pooling layer。防止过拟合，并降维</p>
<p>全连接层：fully-connected layer。汇总前面卷积层和池化层后特征，并输出结果，输出包括：softmax（各个分类的归一化概率分布）、sigmoid（二分类）</p>
<h2 id="基本结构"> 基本结构</h2>
<p>大体上的基本结构主要包括：卷积层、池化层、全连接层</p>
<center><img src="./CNN/architecture-1.png"  width=90% ></center>
<center><img src="./CNN/architecture-2.png"  width=90% ></center>
<h2 id="卷积计算"> 卷积计算</h2>
<center><img src="./CNN/特征提取.gif"  width=50% ></center>
<p>左边动的红色部分是一个不变的3×3的kernel，卷积核。滑动过程中，卷积核对应位置和Image对应位置进行乘积，并求和，得到右边输出结果。</p>
<center><img src="./CNN/卷积.png"  width=50% ></center>
<p>卷积核光顾到的地方称为<strong>感受野</strong>（Receptive Field）。</p>
<div><p>提示</p>
<p>感受野（Receptive Field）：CNN每一层输出的特征图（feature map）上的像素点映射回输入图像上的区域大小。即特征图上一点相对于原图的大小，是CNN特征所能看到输入图像的区域。</p>
<p>感受野作用：</p>
<p>（1）小尺寸的卷积核代替大尺寸的卷积核，可以减少网络参数、增加网络深度、扩大感受野（如3个3×3的卷积核的叠加可以代替7×7的卷积核），网络深度越深，感受野性能越好。</p>
<p>（2）对于分类任务，最后一层feature map感受野大小要≥输入图像大小，否则分类性能不理想。</p>
<p>（3）对于目标检测任务，若感受野小，而目标尺寸大；或感受野大，目标尺寸小，模型收敛困难。</p>
</div>
<p>上图中，左边是原始输入数据，中间是卷积核，右边是输出的新的二维数据，称为<strong>特征图</strong>（feature map），</p>
<center><img src="./CNN/滤波器.jpg"  width=50% ></center>
<p>在CNN中，filter（带一组固定权重的神经元）对局部输入数据进行卷积计算，每计算完一个数据窗口内的局部数据后，数据窗口不断平滑移动，直到计算完所有数据，过程中有如下几个参数：</p>
<div><p>提示</p>
<p>①深度depth：神经元个数，决定输出的depth厚度，也代表滤波器filter个数，也是生成的feature map的个数</p>
<p>②步长stride：决定滑动多少步到边缘。【<strong>有时候需要stride来压缩一部分信息</strong>】</p>
<p>③填充padding：在外围边缘补充若干圈0，方便从初始位置以步长为单位可以刚好滑到末尾位置，即为了总长能被步长整除。使用padding是为了防止边缘特征被忽略。【<strong>有时候需要padding来避免信息损失</strong>】</p>
<p><strong>常用的两种padding方式：</strong></p>
<ul>
<li><strong>valid padding</strong>：不进行处理，只使用原始尺寸，并且不允许卷积核超出原始尺寸边界。</li>
<li><strong>same padding</strong>：进行填充处理，允许卷积核超出原始尺寸边界，并且使得卷积后结果的尺寸和原始输入尺寸一致。</li>
</ul>
</div>
<p><strong>卷积计算</strong>：卷积核和在输入图像上对应滑动到的位置的像素相乘，并求和。</p>
<center><img src="./CNN/卷积计算.png"  width=50% ></center>
<p>假设输入图片的尺寸：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>n</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.4306em;"></span><span>n</span></span></span></span></p>
<p>卷积核大小：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span style="margin-right:0.10764em;">f</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span style="margin-right:0.10764em;">f</span></span></span></span></p>
<p>填充padding大小：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>p</span></span></span></span></p>
<p>步长stride大小：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>s</span></span></span></span></p>
<p>输出图片的尺寸：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>o</span></span></span></span></p>
<p>则输出尺寸的计算公式为：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:2.0574em;vertical-align:-0.7787em;"></span><span><span><span><span><span style="height:1.2787em;"><span style="top:-3.2787em;"><span style="height:3.3714em;"></span><span><span>o</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>⌊</span><span><span></span><span><span><span><span style="height:1.3714em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>s</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>n</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span>2</span><span>p</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span style="margin-right:0.10764em;">f</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span>⌋</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.7787em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:1.2787em;"><span style="top:-3.2787em;"><span style="height:3.3714em;"></span><span></span></span></span><span>​</span></span><span><span style="height:0.7787em;"><span></span></span></span></span></span></span></span></span></p>
<p>其中，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>⌊</span><span>⌋</span></span></span></span>表示向下取整，用于结果不是整数时进行向下取整。</p>
<h2 id="卷积的可视化"> 卷积的可视化</h2>
<blockquote>
<p>一个可视化网站：convolution visualizer：https://ezyang.github.io/convolution-visualizer/index.html</p>
</blockquote>
<h2 id="多通道卷积"> 多通道卷积</h2>
<p>一个可视化例子：https://thomelane.github.io/convolutions/2DConvRGB.html</p>
<p>卷积核红绿蓝部分，分别和输入的红绿蓝层进行做卷积计算，该“三阶魔方”所有结果相加，得到最后的输出。</p>
<p>该部分具体讲解参见：https://www.bilibili.com/video/BV1AJ411Q72b?p=3&amp;spm_id_from=pageDriver</p>
<center><img src="./CNN/可视化过程.gif"  width=80% ></center>
<ul>
<li>左边7×7×3代表图像像素的长宽 ，3是RGB三个颜色通道</li>
<li>中间部分是两个不同的滤波器，是3×3×3的卷积核</li>
<li>两个神经元，depth=2，有两个滤波器</li>
<li>数据窗口每次移动2个步长取3×3的局部数据，stride=2</li>
<li>zero-padding=1</li>
<li>还有个偏置项bias，往往是一个标量，直接相加</li>
</ul>
<p>分别以两个滤波器filter为轴滑动数组进行卷积计算，得到两组不同的结果。</p>
<p>假设stride=1，padding=0，如下图所示：卷积是在提取输入的底层特征。</p>
<center><img src="./CNN/多通道卷积.png"  width=80% ></center>
<p>上图中，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6444em;"></span><span>3</span><span style="margin-right:0.2222em;"></span><span>∗</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>3</span><span style="margin-right:0.2222em;"></span><span>∗</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>3</span></span></span></span>的卷积核和3通道的输入，对应维度进行卷积运算，最后再将三个维度的对应值相加，最后得到一个<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6444em;"></span><span>4</span><span style="margin-right:0.2222em;"></span><span>∗</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>4</span></span></span></span>的矩阵，加上偏置项，可以得到一个最终的单通道的输出。</p>
<p>若存在多个卷积核，把每一个卷积核对应的单通道输出组合成一个总的输出。</p>
<p><strong>filter的通道数和输出的feature map的通道数</strong></p>
<ul>
<li><strong>某一层滤波器的通道数=上一层feature map的通道数</strong>，如上图<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6444em;"></span><span>6</span><span style="margin-right:0.2222em;"></span><span>∗</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>6</span><span style="margin-right:0.2222em;"></span><span>∗</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>3</span></span></span></span>的RGB有3和通道，那么滤波器的也要有3个通道，即3个<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6444em;"></span><span>3</span><span style="margin-right:0.2222em;"></span><span>∗</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>3</span></span></span></span>的矩阵组成一个filter。</li>
<li><strong>某一层feature map的通道数=当前层滤波器的个数</strong>，如上图只有一个filter，输出feature map为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6444em;"></span><span>4</span><span style="margin-right:0.2222em;"></span><span>∗</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>4</span></span></span></span>，只有1个通道；当有2个filter的时候，输出feature map应该是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6444em;"></span><span>4</span><span style="margin-right:0.2222em;"></span><span>∗</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>4</span><span style="margin-right:0.2222em;"></span><span>∗</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>2</span></span></span></span>，通道数为2。</li>
</ul>
<div><p>提示</p>
<p>CNN中的“filter”是“卷积核”的一个集合，在只有一个通道下，卷积核相当于filter，当有多个通道时，filter就包括多个通道里面的卷积核。</p>
<p>真实情况中，卷积核不需要人工设置，而是通过机器学习方法（反向传播，梯度下降方法）学习得到的。</p>
<p>卷积核权重初始值设定，在Keras中为例：<code>kernel_initializer = 'glorot_uniform'</code></p>
</div>
<h2 id="池化层"> 池化层</h2>
<center><img src="./CNN/池化层.gif"  width=60% ></center>
<p>原始图片是20×20的，对其进行下采样，采样窗口是10×10，最终将其下采样成为一个2×2的特征图。这样大大减少运算量，还可以有效的避免过拟合。</p>
<center><img src="./CNN/pooling_type.png"  width=50% ></center>
<ul>
<li>最大池化（Max Pooling）：取区域的最大值。用的最多，</li>
<li>平均池化（Average Pooling）：取区域的平均值</li>
</ul>
<div><p>提示</p>
<p>使用max pooling，舍弃其他值，是否会造成信息损失问题：</p>
<ol>
<li>
<p>CNN中每一个卷积核可以看做一个特征提取器，不同的卷积核负责提取不同的特征，对其进行Max Pooling操作后，提取出的是真正能够识别特征的数值，其余被舍弃的数值，对于所要提取特定的特征并没有特别大的帮助。那么在进行后续计算使，减小了feature map的尺寸，从而减少参数，达到减小计算量。</p>
</li>
<li>
<p>并不是所有情况Max Pooling的效果都很好，有时候有些周边信息也会对某个特定特征的识别产生一定效果，那么这个时候舍弃这部分“不重要”的信息，就不划算了。具体情况具体分析，可以进行一个加与不加的结果输出对比，看最大池化是否有反效果。</p>
</li>
</ol>
</div>
<p><strong>池化的作用</strong>：</p>
<ol>
<li>减少参数量，还保留原始数据特征</li>
<li>防止过拟合</li>
<li>为卷积网络带来平移不变性</li>
</ol>
<p>在线手写数字识别卷积神经网络可视化网站：https://blog.csdn.net/Forrest97/article/details/105632656</p>
<center><img src="./CNN/手写数字识别.png"  width=90% ></center>
<div><p>提示</p>
<p>1、手写字体图片转换成像素矩阵</p>
<p>2、对像素矩阵进行第一层卷积计算，生成六个feature map</p>
<p>3、对每个feature map进行下采样（downsampling），也叫池化（pooling），在保留feature map特征的同时缩小数据量。生成6个小图，这6个小图和上一层各自的feature map很像，但尺寸缩小了</p>
<p>4、对6个小图进行第二层卷积运算，生成更多的feature map</p>
<p>5、对第二次卷积生成的feature map进行下采样</p>
<p>6、第一层全连接层</p>
<p>7、第二层全连接层</p>
<p>8、连接softmax，输出分类结果</p>
</div>
<h2 id="全连接层"> 全连接层</h2>
<p>全连接层：输出的每一个神经元都和上一层的每一个神经元连接。</p>
<center><img src="./CNN/fc.png"  width=20% ></center>
<div><p>提示</p>
<p>卷积核(filter)数量、大小、移动步长（stride）、补0圈数（padding）、全连接层和隐含层层数、神经元个数 都是超参数（hyperparameters），超参数是由人为根据经验指定的。</p>
<p>网络内部权重参数等是由训练得到。</p>
</div>
<p>通过不断的设计卷积核的尺寸，数量，提取更多的特征，最后识别不同类别的物体。做完Max Pooling后，我们就会把这些数据“拍平”，丢到Flatten层，然后把Flatten层的output放到full connected Layer里，采用softmax对其进行分类。</p>
<center><img src="./CNN/全连接层.png"  width=80% ></center>
<p>手写数字可视化识别第一层全连接层：</p>
<center><img src="./CNN/fc-1.jpg"  width=80% ></center>
<p>第二层全连接层：</p>
<center><img src="./CNN/fc-2.jpg"  width=80% ></center>
<p>输出结果：</p>
<center><img src="./CNN/result.jpg"  width=80% ></center>
<h2 id="模型训练过程"> 模型训练过程</h2>
<p>损失函数：</p>
<p>​		计算神经网络的预测结果与图片真实标签的差距，构造损失函数，训练的目标就是将找到损失函数的最小值，最后通过softmax给出预测分类的归一化概率分布决策。</p>
<h2 id="梯度下降"> 梯度下降</h2>
<center><img src="./CNN/梯度下降.png"  width=70% ></center>
<p>要求得当<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.02691em;">w</span></span></span></span>为何值时，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.09618em;">J</span><span>(</span><span style="margin-right:0.02691em;">w</span><span>)</span></span></span></span>能够取得最小值。初始位置的切线的斜率为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span>a</span><span style="margin-right:0.2778em;"></span><span>&gt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span>（即该点导数大于0），<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.02691em;">w</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span style="margin-right:0.02691em;">w</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.4306em;"></span><span>a</span></span></span></span>能使得让<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.02691em;">w</span></span></span></span>的值减小，循环求导更新<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.02691em;">w</span></span></span></span>直到<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.09618em;">J</span><span>(</span><span style="margin-right:0.02691em;">w</span><span>)</span></span></span></span>取得最小值。如果函数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.09618em;">J</span><span>(</span><span style="margin-right:0.02691em;">w</span><span>)</span></span></span></span>包含多个变量，需要分别对不同的变量求<strong>偏导</strong>（partial derivative）来更新不同变量的值。</p>
<h2 id="参考资料"> 参考资料</h2>
<p>1、一位大佬的关于CNN的博客，并用keras做了手写数字识别：</p>
<p>https://towardsdatascience.com/light-on-math-machine-learning-intuitive-guide-to-convolution-neural-networks-e3f054dd5daa</p>
<p>2、同济子豪兄在B站关于CNN的解读：</p>
<p>https://www.bilibili.com/video/BV1AJ411Q72b?p=6&amp;spm_id_from=pageDriver</p>
<p>3、神经网络可视化网站的整理：</p>
<p>https://blog.csdn.net/Forrest97/article/details/105632656</p>
<p>4、一文看懂卷积神经网络，（该网页还包括其它ML算法的解读）：</p>
<p>https://easyai.tech/ai-definition/cnn/</p>
<p>5、大白话讲解CNN工作原理：</p>
<p>https://zhuanlan.zhihu.com/p/49184702</p>
]]></content:encoded>
    </item>
    <item>
      <title>语言模型的简要梳理介绍</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/NLP/LM/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/NLP/LM/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">语言模型的简要梳理介绍</source>
      <description>语言模型（Language Model，LM）的任务是预测下一个句子在语言中出现的概率。到目前，语言模型的发展经历文法规则语言模型、统计语言模型、神经网络语言模型。语言模型可用于提升语音识别和机器翻译的性能。</description>
      <category>自然语言处理</category>
      <pubDate>Sun, 20 Nov 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="语言模型的数学表示"> 语言模型的数学表示</h2>
<p>对于一个由<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>m</span></span></span></span>个<strong>基元</strong>（理解为字、词或短语）构成的句子<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>s</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>，该句子出现概率表示为：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:4.7291em;vertical-align:-2.1145em;"></span><span><span><span><span><span><span style="height:2.6145em;"><span style="top:-5.4259em;"><span style="height:3.6514em;"></span><span><span>p</span><span>(</span><span>s</span><span>)</span></span></span><span style="top:-3.1145em;"><span style="height:3.6514em;"></span><span></span></span></span><span>​</span></span><span><span style="height:2.1145em;"><span></span></span></span></span></span><span><span><span><span style="height:2.6145em;"><span style="top:-5.4259em;"><span style="height:3.6514em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>m</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>)</span></span></span><span style="top:-3.1145em;"><span style="height:3.6514em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∏</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>m</span></span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>)</span></span></span></span><span>​</span></span><span><span style="height:2.1145em;"><span></span></span></span></span></span></span></span></span></span></span></span></p>
<p>其中，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>)</span></span></span></span>表示产生第<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6595em;"></span><span>i</span></span></span></span>个基元（用词代称）的概率由已经产生的前<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7429em;vertical-align:-0.0833em;"></span><span>i</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>1</span></span></span></span>个词<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6389em;vertical-align:-0.2083em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span></span></span></span>决定。</p>
<p>假设一门语言的词汇量为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.22222em;">V</span></span></span></span>，则上述句子<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>s</span></span></span></span>出现概率计算参数量为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span><span style="margin-right:0.22222em;">V</span><span><span><span><span style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>m</span></span></span></span></span></span></span></span></span></span></span>，是一个庞大的数字。</p>
<h2 id="语言模型的评价指标"> 语言模型的评价指标</h2>
<p><strong>困惑度</strong>（perplexity）：在一个测试数据上的perplexity越低，说明语言模型建模效果越好，计算公式如下：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:5.1208em;vertical-align:-2.3104em;"></span><span><span><span><span><span><span style="height:2.8104em;"><span style="top:-5.6855em;"><span style="height:3.8791em;"></span><span><span><span><span style="margin-right:0.01389em;">perplexity</span></span></span><span>(</span><span>s</span><span>)</span></span></span><span style="top:-3.1464em;"><span style="height:3.8791em;"></span><span></span></span></span><span>​</span></span><span><span style="height:2.3104em;"><span></span></span></span></span></span><span><span><span><span style="height:2.8104em;"><span style="top:-5.6855em;"><span style="height:3.8791em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>m</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>)</span><span><span><span><span style="height:1.004em;"><span style="top:-3.413em;margin-right:0.05em;"><span style="height:3em;"></span><span><span><span>−</span><span><span></span><span><span><span><span style="height:0.8443em;"><span style="top:-2.656em;"><span style="height:3em;"></span><span><span><span>m</span></span></span></span><span style="top:-3.2255em;"><span style="height:3em;"></span><span style="border-bottom-width:0.049em;"></span></span><span style="top:-3.384em;"><span style="height:3em;"></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.344em;"><span></span></span></span></span></span><span></span></span></span></span></span></span></span></span></span></span></span></span><span style="top:-3.1464em;"><span style="height:3.8791em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span style="height:0.5762em;"><span style="top:-2.8609em;"><span style="height:2.5em;"></span><span><span><span>m</span></span></span></span></span></span></span></span><span><span><span style="height:1.8791em;"><span style="top:-5.1168em;"><span style="height:5.1168em;"></span><span style="padding-left:1.056em;"><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∏</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>)</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.936em;"><span></span></span></span></span></span><span></span></span></span></span><span style="top:-3.8391em;"><span style="height:5.1168em;"></span><span style="min-width:0.742em;height:3.1968em;"><svg xmlns="http://www.w3.org/2000/svg" width='400em' height='3.1968em' viewBox='0 0 400000 3196' preserveAspectRatio='xMinYMin slice'><path d='M702 80H40000040
H742v3062l-4 4-4 4c-.667.7 -2 1.5-4 2.5s-4.167 1.833-6.5 2.5-5.5 1-9.5 1
h-12l-28-84c-16.667-52-96.667 -294.333-240-727l-212 -643 -85 170
c-4-3.333-8.333-7.667-13 -13l-13-13l77-155 77-156c66 199.333 139 419.667
219 661 l218 661zM702 80H400000v40H742z'/></svg></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:2.3104em;"><span></span></span></span></span></span></span></span></span></span></span></span></p>
<p>perplexity刻画的是一个语言模型预测一个语言样本的能力，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>p</span><span>(</span><span>s</span><span>)</span></span></span></span>越高，语言模型对这个语料库拟合的越好。对于由多个句子构成的测试集$T <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>，通过计算</span></span></span></span>T<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span>中所有的句子概率的乘积来计算</span><span>p</span><span style="margin-right:0.02778em;">er</span><span style="margin-right:0.01968em;">pl</span><span>e</span><span>x</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>，相应地，</span></span></span></span>m$替换为测试集中所有基元的数量。</p>
<blockquote>
<p>如一个由0-9数字随机组成的长度为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>m</span></span></span></span>的序列，由该10个数字出现的概率是随机的，概率为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1901em;vertical-align:-0.345em;"></span><span><span></span><span><span><span><span style="height:0.8451em;"><span style="top:-2.655em;"><span style="height:3em;"></span><span><span><span>10</span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.394em;"><span style="height:3em;"></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.345em;"><span></span></span></span></span></span><span></span></span></span></span></span>，故在任意时刻，模型都有10个等概率的候选答案可用，perplexity为10，即：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:3.4568em;vertical-align:-1.4784em;"></span><span><span><span><span><span><span style="height:1.9784em;"><span style="top:-3.9784em;"><span style="height:3.8791em;"></span><span><span><span><span style="margin-right:0.01389em;">perplexity</span></span></span><span>(</span><span>s</span><span>)</span></span></span></span><span>​</span></span><span><span style="height:1.4784em;"><span></span></span></span></span></span><span><span><span><span style="height:1.9784em;"><span style="top:-3.9784em;"><span style="height:3.8791em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span style="height:0.5762em;"><span style="top:-2.8609em;"><span style="height:2.5em;"></span><span><span><span>m</span></span></span></span></span></span></span></span><span><span><span style="height:1.8791em;"><span style="top:-5.1168em;"><span style="height:5.1168em;"></span><span style="padding-left:1.056em;"><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∏</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>10</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span></span></span><span style="top:-3.8391em;"><span style="height:5.1168em;"></span><span style="min-width:0.742em;height:3.1968em;"><svg xmlns="http://www.w3.org/2000/svg" width='400em' height='3.1968em' viewBox='0 0 400000 3196' preserveAspectRatio='xMinYMin slice'><path d='M702 80H40000040
H742v3062l-4 4-4 4c-.667.7 -2 1.5-4 2.5s-4.167 1.833-6.5 2.5-5.5 1-9.5 1
h-12l-28-84c-16.667-52-96.667 -294.333-240-727l-212 -643 -85 170
c-4-3.333-8.333-7.667-13 -13l-13-13l77-155 77-156c66 199.333 139 419.667
219 661 l218 661zM702 80H400000v40H742z'/></svg></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>10</span></span></span></span><span>​</span></span><span><span style="height:1.4784em;"><span></span></span></span></span></span></span></span></span></span></span></span></p>
</blockquote>
<p>实际语言训练中，通常采用perplexity的<strong>对数</strong>形式表示：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>lo<span style="margin-right:0.01389em;">g</span></span></span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;">(</span><span><span><span style="margin-right:0.01389em;">perplexity</span></span></span><span>(</span><span>s</span><span>)</span><span style="top:0em;">)</span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.9291em;vertical-align:-1.2777em;"></span><span>−</span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>m</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.1667em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span>lo<span style="margin-right:0.01389em;">g</span></span><span style="margin-right:0.1667em;"></span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>)</span></span></span></span></span></p>
<p>将乘积求平方根形式转换为加法，<strong>加速计算</strong>，并且可以<strong>避免乘积数值过小导致的浮点数向下溢出</strong>的问题。</p>
<blockquote>
<p>数学上，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span>lo<span style="margin-right:0.01389em;">g</span></span></span></span></span>形式的perplexity可以看成是<strong>真实分布</strong>和预测分布的交叉熵，也即语言模型优化中往往使用<strong>cross entropy loss</strong>的原因。</p>
<p>真实分布：在测试预料中，认为给定上文<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6389em;vertical-align:-0.2083em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span></span></span></span>的条件下，预料中<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>出现的概率为1，其它词出现概率为0。</p>
</blockquote>
<h2 id="n-gram-model"> n-gram model</h2>
<p>n-gram模型是统计语言模型常用的一种，为解决上述参数呈指数级增长的问题，n-gram model引入了<strong>马尔科夫假设</strong>：当前词的出现频率仅与前<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>n</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>1</span></span></span></span>个词有关。计算公式近似如下：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:7.9581em;vertical-align:-3.7291em;"></span><span><span><span><span><span><span style="height:4.2291em;"><span style="top:-7.0405em;"><span style="height:3.6514em;"></span><span><span>p</span><span>(</span><span>s</span><span>)</span></span></span><span style="top:-4.7291em;"><span style="height:3.6514em;"></span><span></span></span><span style="top:-1.5em;"><span style="height:3.6514em;"></span><span></span></span></span><span>​</span></span><span><span style="height:3.7291em;"><span></span></span></span></span></span><span><span><span><span style="height:4.2291em;"><span style="top:-7.0405em;"><span style="height:3.6514em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>m</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>)</span></span></span><span style="top:-4.7291em;"><span style="height:3.6514em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∏</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>m</span></span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>)</span></span></span><span style="top:-1.5em;"><span style="height:3.6514em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>≐</span><span style="margin-right:0.2778em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∏</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>n</span><span>+</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>)</span></span></span></span><span>​</span></span><span><span style="height:3.7291em;"><span></span></span></span></span></span></span></span></span></span></span></span></p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>n</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8389em;vertical-align:-0.1944em;"></span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>3</span></span></span></span>分别对应<code>unigram</code>，<code>bigram</code>，<code>trigram</code>。理论上<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>的值越大，模型的越准确，但也越复杂，计算量和训练数据量越大。<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7719em;vertical-align:-0.136em;"></span><span>n</span><span style="margin-right:0.2778em;"></span><span>≥</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>4</span></span></span></span>情况很少。</p>
<p>n-gram model一般使用<strong>最大似然估计</strong>（maximum likelihood estimation，MLE）进行参数估计：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:2.363em;vertical-align:-0.9315em;"></span><span><span><span><span><span style="height:1.4315em;"><span style="top:-3.4315em;"><span style="height:3.427em;"></span><span><span>p</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>n</span><span>+</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span></span><span><span><span><span style="height:1.427em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span style="margin-right:0.07153em;">C</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>n</span><span>+</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>)</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span style="margin-right:0.07153em;">C</span><span>(</span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>n</span><span>+</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span></span></span></span><span>​</span></span><span><span style="height:0.936em;"><span></span></span></span></span></span><span></span></span></span></span></span><span>​</span></span><span><span style="height:0.9315em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:1.4315em;"><span style="top:-3.4315em;"><span style="height:3.427em;"></span><span></span></span></span><span>​</span></span><span><span style="height:0.9315em;"><span></span></span></span></span></span></span></span></span></p>
<p>其中，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.07153em;">C</span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span></span></span></span>表示序列<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>在训练预料中出现的次数。训练预料规模越大，参数估计的结果越可靠。</p>
<p>但存在<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.07153em;">C</span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span>的问题，即训练集中，目标序列（如一句话中的某个词）未出现而导致参数为0，进而导致概率为0，引出了<strong>数据平滑</strong>（data  smoothing）策略。</p>
<h2 id="神经网络语言模型"> 神经网络语言模型</h2>
<table>
<thead>
<tr>
<th>模型名称</th>
<th>提出</th>
<th>简述</th>
<th>论文</th>
</tr>
</thead>
<tbody>
<tr>
<td>NNLM</td>
<td>Bengio@2003</td>
<td>将神经网络引入LM中，只能处理定长序列，本质上是一个使用NN编码的n-gram model</td>
<td>《<a href="http://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf" target="_blank" rel="noopener noreferrer">A Neural Probabilistic Language Model</a>》</td>
</tr>
<tr>
<td>RNNLM</td>
<td>Tomas Mikolov@2010</td>
<td>可以处理变长序列，但训练速度太慢，对于大规模的语料库基本无法训练</td>
<td>《<a href="https://www.isca-speech.org/archive/archive_papers/interspeech_2010/i10_1045.pdf" target="_blank" rel="noopener noreferrer">Recurrent Neural Network based  Language Model</a>》</td>
</tr>
<tr>
<td>Word2Vec</td>
<td>Tomas Mikolov@2013</td>
<td>专门用于计算word embedding的工具，提出2个浅层NN模型：CBOW（Continuous Bag-of-Words）、Skip-gram。<br />在隐含层到输出层引入2种优化算法：层次化softmax（Hierarchical Softmax）、负采样（Negative Sampling）</td>
<td>《<a href="https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf" target="_blank" rel="noopener noreferrer">Distributed Representations of Words and Phrases and their Compositionality</a>》<br />《<a href="https://arxiv.org/pdf/1301.3781.pdf" target="_blank" rel="noopener noreferrer">Efficient Estimation of Word Representations in Vector Space</a>》</td>
</tr>
<tr>
<td>GloVe</td>
<td>Jeffrey Pennington@2014</td>
<td>一个基于全局词频统计（count-based&amp; overall statistics）的词表征（word representation）模型。生成的word embedding可以捕捉相似性（similarity）、类比性（analogy）等语义特性</td>
<td>《<a href="https://www.aclweb.org/anthology/D14-1162.pdf" target="_blank" rel="noopener noreferrer">GloVe: Global Vectors for Word Representation</a>》</td>
</tr>
<tr>
<td>ELMo</td>
<td>AllenAI@2018</td>
<td>目的是区分多义词的不同语义</td>
<td>《<a href="https://arxiv.org/pdf/1802.05365.pdf" target="_blank" rel="noopener noreferrer">Deep contextualized word representations</a>》</td>
</tr>
<tr>
<td>GPT</td>
<td>OpenAI@2018</td>
<td>主体采用Transformer Decoder（去除了multi-head attention），训练目标是根据上文预测下文，存在mask机制。<br />通过集合非监督预训练和监督的fine-tuning，为自然语言理解（NLU）任务提供一种普适的半监督学习方式</td>
<td>《<a href="https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf" target="_blank" rel="noopener noreferrer">Improving Language Understanding by Generative Pre-Training</a>》</td>
</tr>
<tr>
<td>BERT</td>
<td>Google@2018</td>
<td>主题采用了Transformer的Encoder，和GPT一样也采用两阶段训练方式：Pre-training和fine-tuning。<br />预训练包括两个无监督任务：Masked LM (MLM)和下一句预测（NSP）<br />fine-tuning只是在预训练模型的output后面增加一个额外的线性层</td>
<td>《<a href="https://arxiv.org/pdf/1810.04805.pdf" target="_blank" rel="noopener noreferrer">BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding</a>》</td>
</tr>
<tr>
<td>GPT-2</td>
<td>OpenAI@2019</td>
<td>相较于GPT，GPT-2提升训练数据的数量、质量和广泛度（webtext）<br />模型结构调整：LN移到每个sub-block的输入位置，最后一个self-attention block后增加一个LN，调整部分参数的初始化方式，扩大词表，seq-len从512提升到1024，batchsize为512<br />GPT-2只保留了pre-training阶段，并直接对zero-shot场景做transfer learning。</td>
<td>《<a href="https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf" target="_blank" rel="noopener noreferrer">Language Models are Unsupervised Multitask Learners</a>》</td>
</tr>
<tr>
<td>ERNIE</td>
<td>Baidu@2019</td>
<td>在BERT的基础进行改进：<br />采用multi-stage的masking策略：Basic-level masking（字粒度）、phrase-level masking（短语粒度）、entity-level masking（实体粒度）<br />预训练阶段采用更丰富的多源语料：中文wiki、百度百科、百度新闻、百度贴吧<br />利用百度贴吧对话数据进行DLM（Dialogue Language Model）任务的训练</td>
<td>《<a href="https://arxiv.org/pdf/1904.09223.pdf" target="_blank" rel="noopener noreferrer">ERNIE: Enhanced Representation through Knowledge Integration</a>》<br />《<a href="https://arxiv.org/pdf/1907.12412.pdf" target="_blank" rel="noopener noreferrer">ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding</a>》</td>
</tr>
<tr>
<td>GPT-3</td>
<td>OpenAI@2020</td>
<td>相较于GPT-2.0，优化点有：<br />数据量提升<br />模型规模提升<br />探讨了在zero-shot、one-shot、few-shot三种不同输入形式下的效果，证明在few-shot下有很好的表现</td>
<td>《<a href="https://arxiv.org/pdf/2005.14165.pdf" target="_blank" rel="noopener noreferrer">Language Models are Few-Shot Learners</a>》</td>
</tr>
<tr>
<td>ChatGPT</td>
<td>OpenAI@2022</td>
<td>是InstructGPT的兄弟模型，模型可以根据输入指令提供详细的回答。通过对话的形式进行问答任务、承认错误、质疑不正确的前提以及拒绝不适当的请求。</td>
<td>《<a href="https://openai.com/blog/chatgpt/" target="_blank" rel="noopener noreferrer">ChatGPT: Optimizing Language Models for Dialogue</a>》</td>
</tr>
</tbody>
</table>
<h2 id="references"> References</h2>
<p>语言模型@https://zhuanlan.zhihu.com/p/90741508</p>
]]></content:encoded>
    </item>
    <item>
      <title>Embedding和word2vec理解</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/NLP/Embedding%E5%92%8Cword2vec/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/NLP/Embedding%E5%92%8Cword2vec/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Embedding和word2vec理解</source>
      <description>Embedding和word2vec理解</description>
      <category>自然语言处理</category>
      <pubDate>Sun, 15 Oct 2023 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="embedding"> Embedding</h2>
<h3 id="概述"> 概述</h3>
<p>Embedding一词广泛出现在自然语言理解、推荐系统、CV等方向。本意是“<strong>嵌入式，嵌入层</strong>”。
通过Embedding，<strong>可以把现实世界中的文字、图片、语言、视频等转化为计算机能识别、能使用的语言，并且转化过程中信息不丢失</strong>。</p>
<p><em>Embedding是用一个</em><strong><em>低维的向量</em></strong><em>表示一个物体</em>（词，商品，可以是万物），Embedding向量的性质使距离相近的向量对应的物体有相近的含义。</p>
<p>比如 Embedding(复仇者联盟)和Embedding(钢铁侠)之间的距离就会很接近，但 Embedding(复仇者联盟)和Embedding(乱世佳人)的距离就会远一些。</p>
<p>Embedding也具有<strong>数学运算关系</strong>，如Embedding（马德里）-Embedding（西班牙）+Embedding(法国)≈Embedding(巴黎)。</p>
<p>Embedding和傅里叶变换有共通之处：<strong>从另一个空间表达物体，甚至揭示物体间的潜在关系</strong>。</p>
<p>Embedding相对于one-hot编码的优势：把物体编码到一个低维稠密向量（并且还能保留其含义 ）喂给DNN，使得更高效。one-hot encoding是一种稀疏向量的编码方式。</p>
<h3 id="深度学习不利于稀疏特征向量处理"> 深度学习不利于稀疏特征向量处理</h3>
<blockquote>
<p><strong>深度学习的结构特点不利于稀疏特征向量的处理：</strong></p>
<p>①神经网络的梯度下降学习过程中，稀疏特征会导致整个网络收敛非常慢，每一个样本的学习只有极少数的权重会更新，在样本数量有限的情况下导致模型不收敛。</p>
<p>②one-hot稀疏特征的维度往往非常大，这样会导致网络的参数量非常庞大，导致很大的算力开销。</p>
</blockquote>
<h2 id="embedding与one-hot"> Embedding与one-hot</h2>
<h3 id="ont-hot编码"> ont-hot编码</h3>
<p>假设如下10个字，分别对应“0-9”表示如下：
<center><img src="./Embedding和word2vec.assets/image1.png" title="BERT结构" width=60% ></center></p>
<p>本质上可以用一个列表就能表示所有对话：
<center><img src="./Embedding和word2vec.assets/image2.png" title="BERT结构" width=60% ></center></p>
<p>但one-hot编码把上面变成：
<center><img src="./Embedding和word2vec.assets/image3.png" title="BERT结构" width=60% ></center></p>
<p>即：把每一个字都对应成一个十个（样本总数/字总数)元素的数组/列表，其中每一个字都用唯一对应的数组/列表对应，数组/列表的唯一性用1表示。</p>
<p>one-hot encoding的维度 = 物体的总数。</p>
<p>对于每一个特征，如果有m个可能值，经过one-hot编码后，变成了m个二元特征，这些特征互斥，每次只有一个激活，<strong>数据变稀疏</strong>。</p>
<p>这样做使得<strong>计算简单</strong>。稀疏矩阵做矩阵计算时，只需要把1对应的位置的数相乘求和即可。</p>
<p>但有明显缺点：<strong>过于稀疏，占用过多资源</strong>。</p>
<h2 id="embedding层"> Embedding层</h2>
<center><img src="./Embedding和word2vec.assets/image4.png" title="BERT结构" width=80% ></center>
<p>Embedding层，某种程度上来说，就是用于<strong>降维</strong>的，本质是<strong>矩阵乘法</strong>。</p>
<blockquote>
<p>一个<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>m</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6944em;"></span><span>d</span></span></span></span>的矩阵乘以<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7778em;vertical-align:-0.0833em;"></span><span>d</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>的矩阵，最终得到一个<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>m</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>的矩阵。</p>
</blockquote>
<p>此外，Embedding层可以进行<strong>升维</strong>。低维空间内可能特征非常“<strong>笼统</strong>”，不便于观察，需要升到高维空间，把其中一些特征进行放大或者把笼统的特征清晰化。</p>
<h2 id="语义理解中的embedding"> 语义理解中的Embedding</h2>
<center><img src="./Embedding和word2vec.assets/image5.png" title="BERT结构" width=80% ></center>
<p>通过将两个无法比较的文字映射成向量，实现计算。</p>
<p>如：</p>
<p>queen = king -man +woman：即皇后是女性的国王。</p>
<p>walked = walking -swimming +swam：即walked是walking的过去式。</p>
<p>另外，向量间的距离也可能会建立联系。</p>
<p>如：<code>|中国|-|北京| = |法国|-|巴黎|</code></p>
<p><strong>通过嵌入的方式，实现两个世界的沟通</strong>。</p>
<blockquote>
<p>例如：</p>
<p>用向量方式从智力层面描述小明。已知：小明的语文成绩88，数学成绩3，英语成绩18，身高149，体重35公斤，父亲是大学教授，母亲是音乐家，立定跳远2.1米，50米自由泳个人记录93秒。</p>
<p>根据已有信息（特征），小明的完整向量表示为：[88, 3, 18, 149, 35, 大学教授, 音乐家, 2.1, 93]。</p>
<p>根据先验知识，身高、体重、立定跳远、游泳成绩、家世和智力无明显关系，摒弃掉，仅保留[语文成绩，数学成绩，英语成绩] 三个维度的特征。</p>
<p>得小明的智力向量: [88, 3, 18]。</p>
<ol>
<li><strong><em>Embedding的数据维度压缩（9维压缩到3维度）</em></strong></li>
<li><strong><em>Embedding通常是丢失信息的——冗余信息和部分有用信息丢失</em></strong></li>
</ol>
</blockquote>
<h2 id="关于embedding的一些观点"> 关于Embedding的一些观点</h2>
<h3 id="embedding本质是one-hot为输入的单层全连接"> Embedding本质是one-hot为输入的单层全连接</h3>
<blockquote>
<p>Embedding的数学本质，是以one-hot为输入的单层全连接。</p>
<p>世界上本没什么Embedding，有的只是one-hot
From:<a href="https://www.zhihu.com/question/374835153" target="_blank" rel="noopener noreferrer">https://www.zhihu.com/question/374835153</a></p>
</blockquote>
<p>Embedding和one-hot的不同，本质而言是矩阵乘法的实现。</p>
<h3 id="embedding分布式表示方法"> Embedding分布式表示方法</h3>
<p>邱锡鹏老师的《神经网络与深度学习》一书。Embedding是一种<strong>分布式表示</strong>方法，即把原始输入数据分布地表示成一系列特征的线性组合。以RGB颜色为例：
<center><img src="./Embedding和word2vec.assets/image6.png" title="BERT结构" width=50% ></center></p>
<p>只需要一个三维的稠密向量就可以表示所有的颜色，并且分布式表示也很容易表示新的颜色。</p>
<p>具体到NLP任务中，词的embedding，每一个句子都被表示成指定维度（如768）的向量，每一个维度对应句子中一个语义特征。</p>
<p>在NLP中，无法从语言学角度先验直到每一个维度（embedding矩阵的列向量）表示哪一种语义特征，也不知道token对应的特征值（token对应句子中的一个词，embedding矩阵的行向量），需要语言模型训练得到。word2vec和BERT也都是设计任务在海量语料上自监督学习得到对应的特征值。
<center><img src="./Embedding和word2vec.assets/image7.png" title="BERT结构" width=40% ></center></p>
<h2 id="word2vec"> word2vec</h2>
<p>Google的word2vec使得embedding方法流行。word2vec是实现<strong>CBOW</strong>（Continuous Bag-of-Words）和<strong>Skip-gram</strong>语言模型的工具。
<center><img src="./Embedding和word2vec.assets/image8.png" width=60% ></center></p>
<p>图片来自：<a href="https://zhuanlan.zhihu.com/p/53194407" target="_blank" rel="noopener noreferrer">https://zhuanlan.zhihu.com/p/53194407</a></p>
<p>CBOW的动机：一个长度为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">T</span></span></span></span>的句子为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>，其中每个词都是由相邻的词决定。</p>
<p>Skip-gram的动机：每个词都决定了相邻的词。</p>
<h3 id="cbow"> CBOW</h3>
<ol>
<li>先是获取大量文本数据</li>
<li>后我们建立一个可以沿文本滑动的窗(例如一个窗里包含三个单词)</li>
<li>利用这样的滑动窗就能为训练模型生成大量样本数据。</li>
</ol>
<center><img src="./Embedding和word2vec.assets/image9.png" width=60% ></center>
<p>通过滑动窗产生一个个样本，最终得到一个很大的数据集。</p>
<p>考虑目标单词（red）前后两个单词的情况：</p>
<center><img src="./Embedding和word2vec.assets/image10.png" width=60% ></center>
<p>最终得到的形式：
<center><img src="./Embedding和word2vec.assets/image11.png" width=60% ></center></p>
<p>图片摘自：<a href="https://zhuanlan.zhihu.com/p/269312855" target="_blank" rel="noopener noreferrer">https://zhuanlan.zhihu.com/p/269312855</a></p>
<h3 id="skip-gram"> Skip-gram</h3>
<center><img src="./Embedding和word2vec.assets/image12.png" width=60% ></center>
<p>其中绿框是输入词，粉狂是可能的输出结果，上图中滑动窗给训练集产生了4个独立样本，即：
<center><img src="./Embedding和word2vec.assets/image13.png" width=60% ></center></p>
<p>移动几组，得到一批样本：
<center><img src="./Embedding和word2vec.assets/image14.png" width=80% ></center></p>
<h3 id="skip-gram-with-negative-sampling"> Skip-gram with Negative Sampling</h3>
<center><img src="./Embedding和word2vec.assets/image15.png" width=60% ></center>
<p>上面模型执行三个步骤并输出预测向量，模型未训练，预测结果是错误的。</p>
<p>针对输入句子：Thou shalt not make a machine in the likeness of a human mind.</p>
<center><img src="./Embedding和word2vec.assets/image16.png" width=60% ></center>
<p>输入单词（not）的目标单词是thou，其概率为1，其它所有单词概率为0，这样组成了“目标向量”。将模型预测（model prediction）和目标向量（actual target）相减得到偏差向量。</p>
<p>偏差向量被用于模型更新：
<center><img src="./Embedding和word2vec.assets/image17.png" width=60% ></center></p>
<p>这样下去遍历所有的样本（一个epoch），几轮epoch之后，得到训练好的模型，从中提取<strong>嵌入矩阵</strong>（embedding matrix）用于下游任务。</p>
<p>上面涉及到<strong>负采样</strong>（negative sampling)的概念：</p>
<p>在数据集中随机引入负样本——不是邻居的单词样本，并且这些样本的概率值为0。这些样本是从vocab词表中随机抽取，模拟噪声对比估计。</p>
<h2 id="word2vec训练与embedding矩阵"> word2vec训练与embedding矩阵</h2>
<center><img src="./Embedding和word2vec.assets/image18.png" width=60% ></center>
<p>训练开始阶段，创建上述两个矩阵：</p>
<p>Embedding矩阵中嵌入了词表（vocab）中的每个单词，维度之一是vocab_size，第二个维度是每一次嵌入的长度（embedding_size），这个是设定值。【Embedding矩阵作为网络的Embedding层参数，跟随网络一起学习训练。】</p>
<p>Context矩阵是放入了词表中的每个单词。</p>
<center><img src="./Embedding和word2vec.assets/image19.png" width=60% ></center>
<p>上图展示：输入单词（not）和输出的上下文单词：thou（实际邻居词）、aaron和taco（引入的负样本)。</p>
<ol>
<li>对于输入词，查找embedding矩阵中对应的向量</li>
<li>对于上下文词，查找Context矩阵对应的向量</li>
<li>计算输入和每个上下文嵌入向量的<strong>点积【余弦相似度】</strong>。并使用sigmoid激活函数，得到0-1之间的概率值。计算出误差error。</li>
<li>利用误差分数来调整not、thou、aaron和taco的嵌入。</li>
<li>多次迭代epoch训练后，Embedding嵌入矩阵得到改进，丢弃Context矩阵，使用Embedding矩阵用于下游任务</li>
</ol>
<center><img src="./Embedding和word2vec.assets/image20.png" width=60% ></center>
<p>【点积计算两者的相似度，word2vec运用的是<strong>余弦相似度</strong>】</p>
<blockquote>
<p><strong>cosine similarity</strong>：内积空间的两个非零向量之间相似度的量度</p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span><span style="margin-right:0.01389em;">similarity</span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>cos</span><span>(</span><span style="margin-right:0.02778em;">θ</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.3923em;vertical-align:-0.52em;"></span><span><span></span><span><span><span><span style="height:0.8723em;"><span style="top:-2.655em;"><span style="height:3em;"></span><span><span><span>∣∣</span><span><span><span><span><span style="height:0.6833em;"></span><span><span><span>A</span></span></span><span></span></span></span></span><span style="margin-right:0.0717em;"></span><span>A</span></span><span>∣∣</span><span>⋅</span><span>∣∣</span><span><span><span><span><span style="height:0.6833em;"></span><span><span><span style="margin-right:0.05017em;">B</span></span></span><span></span></span></span></span><span style="margin-right:0.0717em;"></span><span style="margin-right:0.05017em;">B</span></span><span>∣∣</span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.394em;"><span style="height:3em;"></span><span><span><span><span><span><span><span style="height:0.6833em;"></span><span><span><span>A</span></span></span><span></span></span></span></span><span style="margin-right:0.0717em;"></span><span>A</span></span><span>⋅</span><span><span><span><span><span style="height:0.6833em;"></span><span><span><span style="margin-right:0.05017em;">B</span></span></span><span></span></span></span></span><span style="margin-right:0.0717em;"></span><span style="margin-right:0.05017em;">B</span></span></span></span></span></span><span>​</span></span><span><span style="height:0.52em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:3.641em;vertical-align:-1.6296em;"></span><span><span></span><span><span><span><span style="height:2.0114em;"><span style="top:-2.19em;"><span style="height:3.0805em;"></span><span><span><span><span><span><span style="height:1.5436em;"><span style="top:-4.5714em;"><span style="height:4.5714em;"></span><span style="padding-left:1.4286em;"><span><span><span><span style="height:1.4005em;"><span style="top:-1.8629em;margin-left:0em;"><span style="height:2.75em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-2.75em;"><span style="height:2.75em;"></span><span><span>∑</span></span></span><span style="top:-3.7em;margin-left:0em;"><span style="height:2.75em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.0301em;"><span></span></span></span></span></span><span style="margin-right:0.1952em;"></span><span><span>A</span><span><span><span><span style="height:0.3281em;"><span style="top:-2.357em;margin-left:0em;margin-right:0.0714em;"><span style="height:2.5em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.143em;"><span></span></span></span></span></span></span></span></span><span style="top:-3.5156em;"><span style="height:4.5714em;"></span><span style="min-width:1.02em;height:2.6857em;"><svg xmlns="http://www.w3.org/2000/svg" width='400em' height='2.6857em' viewBox='0 0 400000 1944' preserveAspectRatio='xMinYMin slice'><path d='M983 90
l0 -0
c4,-6.7,10,-10,18,-10 H400000v40
H1013.1s-83.4,268,-264.1,840c-180.7,572,-277,876.3,-289,913c-4.7,4.7,-12.7,7,-24,7
s-12,0,-12,0c-1.3,-3.3,-3.7,-11.7,-7,-25c-35.3,-125.3,-106.7,-373.3,-214,-744
c-10,12,-21,25,-33,39s-32,39,-32,39c-6,-5.3,-15,-14,-27,-26s25,-30,25,-30
c26.7,-32.7,52,-63,76,-91s52,-60,52,-60s208,722,208,722
c56,-175.3,126.3,-397.3,211,-666c84.7,-268.7,153.8,-488.2,207.5,-658.5
c53.7,-170.3,84.5,-266.8,92.5,-289.5z
M1001 80h400000v40h-400000z'/></svg></span></span></span><span>​</span></span><span><span style="height:1.0559em;"><span></span></span></span></span></span><span><span><span><span style="height:1.5436em;"><span style="top:-4.5714em;"><span style="height:4.5714em;"></span><span style="padding-left:1.4286em;"><span><span><span><span style="height:1.4005em;"><span style="top:-1.8629em;margin-left:0em;"><span style="height:2.75em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-2.75em;"><span style="height:2.75em;"></span><span><span>∑</span></span></span><span style="top:-3.7em;margin-left:0em;"><span style="height:2.75em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.0301em;"><span></span></span></span></span></span><span style="margin-right:0.1952em;"></span><span><span style="margin-right:0.05017em;">B</span><span><span><span><span style="height:0.3281em;"><span style="top:-2.357em;margin-left:-0.0502em;margin-right:0.0714em;"><span style="height:2.5em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.143em;"><span></span></span></span></span></span></span></span></span><span style="top:-3.5156em;"><span style="height:4.5714em;"></span><span style="min-width:1.02em;height:2.6857em;"><svg xmlns="http://www.w3.org/2000/svg" width='400em' height='2.6857em' viewBox='0 0 400000 1944' preserveAspectRatio='xMinYMin slice'><path d='M983 90
l0 -0
c4,-6.7,10,-10,18,-10 H400000v40
H1013.1s-83.4,268,-264.1,840c-180.7,572,-277,876.3,-289,913c-4.7,4.7,-12.7,7,-24,7
s-12,0,-12,0c-1.3,-3.3,-3.7,-11.7,-7,-25c-35.3,-125.3,-106.7,-373.3,-214,-744
c-10,12,-21,25,-33,39s-32,39,-32,39c-6,-5.3,-15,-14,-27,-26s25,-30,25,-30
c26.7,-32.7,52,-63,76,-91s52,-60,52,-60s208,722,208,722
c56,-175.3,126.3,-397.3,211,-666c84.7,-268.7,153.8,-488.2,207.5,-658.5
c53.7,-170.3,84.5,-266.8,92.5,-289.5z
M1001 80h400000v40h-400000z'/></svg></span></span></span><span>​</span></span><span><span style="height:1.0559em;"><span></span></span></span></span></span></span></span></span><span style="top:-3.3105em;"><span style="height:3.0805em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-4.1116em;"><span style="height:3.0805em;"></span><span><span><span><span><span><span style="height:1.4005em;"><span style="top:-1.8629em;margin-left:0em;"><span style="height:2.75em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-2.75em;"><span style="height:2.75em;"></span><span><span>∑</span></span></span><span style="top:-3.7em;margin-left:0em;"><span style="height:2.75em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.0301em;"><span></span></span></span></span></span><span style="margin-right:0.1952em;"></span><span><span>A</span><span><span><span><span style="height:0.3281em;"><span style="top:-2.357em;margin-left:0em;margin-right:0.0714em;"><span style="height:2.5em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.143em;"><span></span></span></span></span></span></span><span><span style="margin-right:0.05017em;">B</span><span><span><span><span style="height:0.3281em;"><span style="top:-2.357em;margin-left:-0.0502em;margin-right:0.0714em;"><span style="height:2.5em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.143em;"><span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:1.6296em;"><span></span></span></span></span></span><span></span></span></span></span></span></p>
<center><img src="./Embedding和word2vec.assets/image21.png"  width=80% ></center>
</blockquote>
<p>上述训练有两个关键的超参数：<strong>滑动窗口大小</strong>和<strong>负样本的数量</strong>。</p>
<p>窗口大小选择的一个启发方式：</p>
<p>较小窗口（2-15）得到：高相似得分表明两个词可互换（距离近）。</p>
<p>较大窗口（15-50）得到：相似性更能指示单词相关性的嵌入。</p>
<p>Gnesim默认窗口大小为5，默认的负采样样本数为5，当数据集足够大时，2-5个负采样样本数也能达到理想效果。
<center><img src="./Embedding和word2vec.assets/image22.png" width=80% ></center></p>
<h2 id="参考教程"> 参考教程</h2>
<p>1、Embedding的概念，以及它和深度学习的关系：<a href="https://zhuanlan.zhihu.com/p/164502624" target="_blank" rel="noopener noreferrer">https://zhuanlan.zhihu.com/p/164502624</a></p>
<p>2、Embedding和word2vec理解：<a href="https://zhuanlan.zhihu.com/p/269312855" target="_blank" rel="noopener noreferrer">https://zhuanlan.zhihu.com/p/269312855</a></p>
<p>3、万物皆Embedding，从经典的word2vec到深度学习基本操作item2vec：<a href="https://zhuanlan.zhihu.com/p/53194407" target="_blank" rel="noopener noreferrer">https://zhuanlan.zhihu.com/p/53194407</a></p>
]]></content:encoded>
    </item>
    <item>
      <title>Transformer-Mask机制理解</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/NLP/Transformer-Mask%E6%9C%BA%E5%88%B6/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/NLP/Transformer-Mask%E6%9C%BA%E5%88%B6/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Transformer-Mask机制理解</source>
      <description>Transformer-Mask机制理解</description>
      <category>自然语言处理</category>
      <pubDate>Sun, 15 Oct 2023 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<center><img src="./Transformer-Mask机制.assets/image1.png" width=40% ></center>
<p>图片摘自：<a href="https://ifwind.github.io/2021/08/17/Transformer%E7%9B%B8%E5%85%B3%E2%80%94%E2%80%94%EF%BC%887%EF%BC%89Mask%E6%9C%BA%E5%88%B6/" target="_blank" rel="noopener noreferrer">博客文章</a></p>
<p>Mask（掩码）机制经常用于NLP任务中，大致可以分为两类：</p>
<p>（1）用于处理非定长序列的Padding的Mask；去除各种Padding在训练过程中的影响。【作用Encoder】</p>
<p>（2)用于防止标签泄露的Sequence的Mask；将输入进行遮盖，避免decoder看到后面要预测的东西。【作用Decoder】</p>
<h2 id="padding-mask"> Padding Mask</h2>
<p>在NLP任务中，文本通常是不定长的，所以在输入一个样本长短不一的batch到网络前，要对batch中的样本进行truncating截断/padding补齐操作，以便能形成一个张量的形式输入网络：
<center><img src="./Transformer-Mask机制.assets/image2.png" width=60% ></center>
Mask矩阵中1代表有效数字，0代表无效字（或者使用T/F）。</p>
<p><strong>Padding Mask的生成</strong>：padding补齐操作在batch输入网络前完成，并同步生成Mask矩阵。</p>
<p><strong>Padding Mask的使用</strong>：通常用在最终结果输出、损失函数计算等；即不需要无用的padding参与计算时。</p>
<p><strong>在self-attention中的padding mask</strong></p>
<div><pre><code><span>def</span> <span>attention</span><span>(</span>query<span>,</span> key<span>,</span> value<span>,</span> mask<span>=</span><span>None</span><span>,</span> dropout<span>=</span><span>None</span><span>)</span><span>:</span>
    <span>"Compute 'Scaled Dot Product Attention'"</span>
    d_k <span>=</span> query<span>.</span>size<span>(</span><span>-</span><span>1</span><span>)</span>
    scores <span>=</span> torch<span>.</span>matmul<span>(</span>query<span>,</span> key<span>.</span>transpose<span>(</span><span>-</span><span>2</span><span>,</span> <span>-</span><span>1</span><span>)</span><span>)</span> <span>/</span> math<span>.</span>sqrt<span>(</span>d_k<span>)</span>
    
    <span>#Q,K点积之后，需要先经过mask再进行softmax</span>
    <span>#对于要屏蔽的部分，mask之后的输出需要为负无穷（极小值），这样softmax后输出为0，避免padding的影响</span>
    <span>if</span> mask <span>is</span> <span>not</span> <span>None</span><span>:</span>
        scores <span>=</span> scores<span>.</span>masked_fill<span>(</span>mask <span>==</span> <span>0</span><span>,</span> <span>-</span><span>1e9</span><span>)</span> <span># mask步骤，用 -1e9 代表负无穷</span>
    p_attn <span>=</span> F<span>.</span>softmax<span>(</span>scores<span>,</span> dim <span>=</span> <span>-</span><span>1</span><span>)</span>
    
    <span>if</span> dropout <span>is</span> <span>not</span> <span>None</span><span>:</span>
        p_attn <span>=</span> dropout<span>(</span>p_attn<span>)</span>
    <span>return</span> torch<span>.</span>matmul<span>(</span>p_attn<span>,</span> value<span>)</span><span>,</span> p_attn
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br></div></div><h2 id="sequence-mask"> Sequence Mask</h2>
<p>常见的应用场景是：需要一个词预测下一个词。若使用self attention或者其它使用上下文信息的机制，会导致模型“提前看到”待预测的内容。
为不泄露要预测的标签信息，需要Mask来“遮盖”。</p>
<center><img src="./Transformer-Mask机制.assets/image3.png" width=60% ></center>
<h2 id="bert和ernie中的mask"> BERT和ERNIE中的Mask</h2>
<p>在decoder部分加入mask除防止标签泄露以外，模型利用这种填空机制可以帮助模型学习更好。如BERT和ERNIE模型中利用的Masked LM（MLM）。</p>
<p>Masked LM随机掩盖部分输入词，然后对那些被掩盖的词进行预测。在训练的过程中，BERT随机地掩盖每个序列中15％的token，并不是像word2vec中的cbow那样去对每一个词都进行预测。MLM从输入中随机地掩盖一些词，其目标是基于其上下文来预测被掩盖单词的原始词汇。</p>
<p>ERINE不是在token level进行mask操作，而是在phrase level进行mask操作。</p>
<center><img src="./Transformer-Mask机制.assets/image4.png" width=60% ></center>
<p>图片摘自：<a href="https://ifwind.github.io/2021/08/17/Transformer%E7%9B%B8%E5%85%B3%E2%80%94%E2%80%94%EF%BC%887%EF%BC%89Mask%E6%9C%BA%E5%88%B6/" target="_blank" rel="noopener noreferrer">博客文章</a></p>
<h2 id="参考资料"> 参考资料</h2>
<p>1、冬于博客——<a href="https://ifwind.github.io/2021/08/17/Transformer%E7%9B%B8%E5%85%B3%E2%80%94%E2%80%94%EF%BC%887%EF%BC%89Mask%E6%9C%BA%E5%88%B6/" target="_blank" rel="noopener noreferrer">Transformer的Mask机制</a></p>
<p>2、Transformer中的mask：<a href="https://blog.csdn.net/weixin_42253689/article/details/113838263" target="_blank" rel="noopener noreferrer">https://blog.csdn.net/weixin_42253689/article/details/113838263</a></p>
]]></content:encoded>
    </item>
    <item>
      <title>自然语言处理基础知识</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/NLP/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/NLP/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">自然语言处理基础知识</source>
      <description>整理的关于在自然语言处理领域的基础知识。</description>
      <content:encoded><![CDATA[<p>BERT预训练模型</p>
<p>语言模型</p>
<p>扩散模型</p>
<p>。。。</p>
]]></content:encoded>
    </item>
    <item>
      <title>BERT预训练模型</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/NLP/bert/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/NLP/bert/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">BERT预训练模型</source>
      <description>BERT预训练模型原理分析</description>
      <category>自然语言处理</category>
      <pubDate>Tue, 15 Nov 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="introduction"> Introduction</h2>
<p>BERT（Bidirectional Encoder Representations from Transformers）是一个新的语言表达模型，设计被用于预训练的未标记文本深度双向表示，作用于所有层的左右上下文。预训练的BERT只需要一个额外的输出层就可以实现模型微调，针对问答（question answering）和语言推理（language inference）等问题，不需要进行大量实质性的模型修改。</p>
<h2 id="language-model"> Language Model</h2>
<p>语言模型预训练已经被证明可以有效地提高许多自然语言处理任务的性能。</p>
<table>
<thead>
<tr>
<th>任务</th>
<th>层次</th>
</tr>
</thead>
<tbody>
<tr>
<td>token-level</td>
<td>命名实体识别（NER）、SQuAD问答任务等</td>
</tr>
<tr>
<td>sentence-level</td>
<td>自然语言推断（NLI）、情感识别、句子语义等价判断等</td>
</tr>
</tbody>
</table>
<p>针对自然语处理的下游任务（downstream-task），预训练的语言模型发挥重要作用。现有的语言模型（LM）应用到NLP任务主要有两种策略：feature-based和fine-tuning<sup></sup>。</p>
<table>
<thead>
<tr>
<th>feature-based</th>
<th>fine-tuning</th>
</tr>
</thead>
<tbody>
<tr>
<td>ELMo (<em>Peters et al., 2018</em>)</td>
<td>OpenAI GPT (<em>Radford et al., 2018</em>)</td>
</tr>
</tbody>
</table>
<p>BERT是基于fine-tuning预训练方法的改进，提出了Masked Language Model（<strong>MLM</strong>）实现深度双向表示（deep bidirectional representation）的预训练；此外，引入了Next Sentence Prediction（<strong>NSP</strong>）任务，目的是让模型能够更好地理解句子间的关系。</p>
<h2 id="bert-architecture"> BERT Architecture</h2>
<center><img src="./bert/architecture.png" title="BERT结构" width=80% ></center>
<p>BERT的结构是一个多层的双向的Transformer Encoder，模型要比Transformer深。</p>
<table>
<thead>
<tr>
<th><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span><span><span>BERT</span></span></span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span><span>BASE</span></span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span></th>
<th><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span><span><span>BERT</span></span></span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span><span>LARGE</span></span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span></th>
<th>Transformer</th>
</tr>
</thead>
<tbody>
<tr>
<td>层数：Transformer blocks (<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>L</span></span></span></span>)=12</td>
<td>Transformer blocks (<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>L</span></span></span></span>)=24</td>
<td>Encode block = 6</td>
</tr>
<tr>
<td>Hidden size (<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.08125em;">H</span></span></span></span>) = 768</td>
<td>Hidden size (<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.08125em;">H</span></span></span></span>) = 1024</td>
<td></td>
</tr>
<tr>
<td>feed-forward/filter size = <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7278em;vertical-align:-0.0833em;"></span><span>4</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.08125em;">H</span></span></span></span> =3072</td>
<td>feed-forward/filter size = <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7278em;vertical-align:-0.0833em;"></span><span>4</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.08125em;">H</span></span></span></span> =4096</td>
<td></td>
</tr>
<tr>
<td>自注意力头数：self-attention heads (<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>) =12</td>
<td>self-attention heads (<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>) =16</td>
<td></td>
</tr>
<tr>
<td>Total parameters = 110M</td>
<td>Total parameters = 340M</td>
<td></td>
</tr>
</tbody>
</table>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span><span><span>BERT</span></span></span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span><span>BASE</span></span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>模型的尺寸和OpenAI GPT保持一致便于性能对比，唯一差别是BERT Transfomer使用的是bidirectional self-attention （也被称为<strong>Transformer Encoder</strong>）。</p>
<table>
<thead>
<tr>
<th>模型</th>
<th>左右上下文语义</th>
<th>是否并行</th>
</tr>
</thead>
<tbody>
<tr>
<td>Word2Vec</td>
<td>True</td>
<td>True</td>
</tr>
<tr>
<td>ELMo</td>
<td>True (<strong>shallow bidirectional</strong>)</td>
<td>False</td>
</tr>
<tr>
<td>OpenAI GPT</td>
<td>False</td>
<td>True</td>
</tr>
<tr>
<td>BERT</td>
<td>True (<strong>deep bidirectional</strong>)</td>
<td>True</td>
</tr>
</tbody>
</table>
<p><strong>OpenAI GPT</strong>使用的是constrained self-attention，每一个token只关注左侧的上下文信息，left-context-only版本的self-attention也被称为<strong>Transformer Decoder</strong>。</p>
<p>上图中的<strong>ELMo模型</strong>使用的是<em>concatenation of independency trained left-to-right and right-to-left LSTM</em>来生成用于下游任务的特征，称为<strong>浅层双向模型</strong>（shallow bidirectional model）。</p>
<p><strong>Word2Vec的CBOW</strong>模型是通过token的上文信息和下文信息来预测该token，使用的是词袋模型，不知道单词的顺序信息，相当于直接把该token给mask了。</p>
<p><strong>BERT</strong>可以将每一个token的左边上下文信息和右边的上下文信息结合起来，这样可以更好地根据全文理解每一个token的意思，训练一个<strong>深度双向模型</strong>（deep bidirectional model）</p>
<h2 id="bert-input-representation"> BERT Input Representation</h2>
<center><img src="./bert/input.png" title="BERT的输入表示" width=60%></center>
<p>BERT支持一个句子对输入（句子A和句子B）或单个句子的输入，BERT的输入会在分词的句子里面增加有特殊作用的标志位：</p>
<ul>
<li>
<p><code>[CLS]</code>放在句首，classification，用于下游的分类任务。该符号对应的输出向量作为整个输入文本的语义表示，用于文本分类。</p>
<ul>
<li><strong>单句子任务</strong>：如情感分析，<code>[CLS]</code>符号对于的输出向量作为整个输入文本的语义表示，用于文本分类。</li>
<li><strong>语句对分类任务</strong>：包括问答（判断问题和答案是否匹配）、语句匹配（两句话是否表达同一个意思）等，使用<code>[CLS]</code>作为文本的语义表示，同时也利用<code>[SEP]</code>作为分割标识符，分别对两句话附加两个不同的文本向量作为区分。</li>
</ul>
</li>
<li>
<p><code>[SEP]</code>用于分开两个输入的句子（A和B），分别在句子A和B的最后增加该标志</p>
</li>
<li>
<p><code>[UNK]</code>表示未知字符</p>
</li>
<li>
<p><code>[MASK]</code>用于遮盖句子中的一些单词，将单词用MASK遮盖后，根据上下文语境，利用BERT输出的[MASK]向量预测单词是什么。</p>
</li>
</ul>
<p>BERT得到要输入的句子进行分词<strong>tokenizer</strong>得到token，然后将句子的token转成<strong>Embedding</strong>，用<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05764em;">E</span></span></span></span>表示。</p>
<div><p>提示</p>
<p>BERT使用一个词表文件<code>vocab.txt</code>作为对输入进行分词的一个依据，对输入文本进行<code>split</code>操作后得到token。BERT包含三个tokenizer方式：</p>
<ul>
<li>
<p><strong>BasicsTokenizer</strong>：清理特殊字符，包括控制符以及替换空白字符为空格。</p>
</li>
<li>
<p><strong>WordpieceTokenier</strong>：对输入进行<code>split</code>，得到token的列表。如果token的一部分仍然在<code>vocab.txt</code>中，则继续分词，分割后除去第一部分外，其他部分都会加上<code>##</code>，得到WordPiece。如<code>playing</code><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.3669em;"></span><span>→</span></span></span></span><code>play</code><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>+</span></span></span></span><code>##ing</code>。</p>
</li>
<li>
<p><strong>FullTokenier</strong>：调用BasicTokenier得到初步的token列表，对于token再调用WordpieceTokenizer，得到更细分的token。</p>
</li>
</ul>
</div>
<p>BERT的输入Embedding包括三个部分构成：Token Embedding，Segment Embedding，Position Embedding。</p>
<ul>
<li>
<p><strong>Token Embedding</strong>：通过训练学习得到，如上图中的<code>[CLS]</code>,<code>dog</code>等。</p>
</li>
<li>
<p><strong>Segment Embedding</strong>：用于区分每一个单词属于句子A或B，如果只有一个句子，用<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.05764em;">E</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0576em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>表示，通过学习得到</p>
</li>
<li>
<p><strong>Position Embedding</strong>：编码单词出现的位置，通过学习得到。在BERT中，默认的句子最大长度为<strong>512</strong>。</p>
</li>
</ul>
<h2 id="pre-training-tasks"> Pre-training Tasks</h2>
<p>BERT预训练模型包含两个任务：<strong>Masked LM</strong>和<strong>NSP</strong>。</p>
<h3 id="masked-lm任务"> Masked LM任务</h3>
<p>在句子中随机用<code>[MASK]</code>替换一部分单词，然后将句子传入BERT模型编码每一个token的信息，最终用<code>[MASK]</code>的编码信息<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span><span><span>T</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>MASK</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span></span></span>预测该位置正确的token。</p>
<center><img src="./bert/pre-training.png" title="BERT预训练过程" width=70%></center>
<p>图源<sup></sup>。</p>
<p>在原文<sup></sup>中，随机mask了每一个sentence的全部WordPiece token的15%。</p>
<p>为了减缓<strong>错配</strong>的发生，并不是所有的待遮盖的token都会用<code>[MASK]</code>符号代替，在上述任意选择的15%待遮盖的token中，再任意选择其中的80%用<code>[MASK]</code>代替，10%用任意的其他随机token取代（i.e., 被随机取代的token占总的token的1.5%），剩余的10%保持不变。这样Transformer Encoder就无法知道哪一个token是要被预测，哪一个token被其他任意单词取代。</p>
<p>每一个batch的15%的token会被预测，这样会导致MLM需要<strong>更多的epoch才能达到收敛</strong>。文章也说明了花费相对较长一些的computational cost获得更高的预测准确率是值得的。</p>
<div><p>提示</p>
<p>在语言模型（LM）中，常需要用上一个词预测下一个词，BERT中融合了左右两侧上下文信息实现预训练深度双向Transformer，要使得在<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6151em;"></span><span>t</span></span></span></span>时刻看不到<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6151em;"></span><span>t</span></span></span></span>时刻之后的信息，需要用<code>[MASK]</code>来“遮盖”。</p>
<center><img src="./bert/MLM.png" title="BERT的Masked LM" width=50%></center>
<p>图源<sup></sup>。</p>
</div>
<h3 id="nsp任务"> NSP任务</h3>
<p>下一句预测是将句子<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>和<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">B</span></span></span></span>输入BERT，预测<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">B</span></span></span></span>是否是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>的下一句，使用<code>[CLS]</code>的编码信息<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07153em;">C</span></span></span></span>进行预测。</p>
<p>NSP应用的下游任务场景包括：Question Answering (QA)，Natural Language Inference (NLI)等，主要解决的是理解<strong>两个文本句子</strong>之间的关系。</p>
<p>原文<sup></sup>中的预训练样本，有50%的sentence pair是连续相连的，另外50%的是从语料库中随机选择不连续的。然后通过<code>[CLS]</code>标志位的输出<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07153em;">C</span></span></span></span>预测句子<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>的下一句<code>IsNext</code> or <code>NotNext</code>是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">B</span></span></span></span>。</p>
<h2 id="pre-training-procedure"> Pre-training Procedure</h2>
<table>
<thead>
<tr>
<th>Hyperparameter</th>
<th>value</th>
</tr>
</thead>
<tbody>
<tr>
<td>batch size</td>
<td><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6444em;"></span><span>256</span></span></span></span> sequences=<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7278em;vertical-align:-0.0833em;"></span><span>256</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>512</span></span></span></span> tokens=<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8389em;vertical-align:-0.1944em;"></span><span>128</span><span>,</span><span style="margin-right:0.1667em;"></span><span>000</span></span></span></span> tokens/batch</td>
</tr>
<tr>
<td>epoch</td>
<td>40 epoches for 1M steps</td>
</tr>
<tr>
<td>word corpus size</td>
<td>3.3 billion</td>
</tr>
<tr>
<td>Optimizer</td>
<td>Adam</td>
</tr>
<tr>
<td>learning rate</td>
<td>warmup over the first 10000 steps at <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7278em;vertical-align:-0.0833em;"></span><span>1</span><span>e</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>4</span></span></span></span>  and linear decay after 10000 steps</td>
</tr>
<tr>
<td>Dropout</td>
<td>0.1 on all layers</td>
</tr>
<tr>
<td>activation function</td>
<td>gelu</td>
</tr>
<tr>
<td>training loss</td>
<td>MLM似然值的均值和NSP似然值的均值的和</td>
</tr>
</tbody>
</table>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span><span>BER</span><span><span>T</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>BASE</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span></span></span>用到了16个TPU（张量处理器），<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span><span>BER</span><span><span>T</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>LARGE</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span></span></span>用到了64个TPU，每一次训练时间4 days。</p>
<h2 id="fine-tuning-procedure"> Fine-tuning Procedure</h2>
<p>对于序列级（sequence-level）的分类任务，BERT将最终的隐藏层输出（i.e., Transformer的输出）给第一个token（i.e., <code>[CLS]</code>）的word embedding。<code>[CLS]</code>标志位的输出<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7224em;vertical-align:-0.0391em;"></span><span style="margin-right:0.07153em;">C</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8413em;"></span><span><span>R</span><span><span><span><span style="height:0.8413em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.08125em;">H</span></span></span></span></span></span></span></span></span></span></span>，利用softmax输出标签概率<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">P</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.0913em;vertical-align:-0.25em;"></span><span><span><span>softmax</span></span></span><span>(</span><span style="margin-right:0.07153em;">C</span><span><span style="margin-right:0.13889em;">W</span><span><span><span><span style="height:0.8413em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span><span>)</span></span></span></span>，其中<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7224em;vertical-align:-0.0391em;"></span><span style="margin-right:0.13889em;">W</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8413em;"></span><span><span>R</span><span><span><span><span style="height:0.8413em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span style="margin-right:0.07153em;">K</span><span>×</span><span style="margin-right:0.08125em;">H</span></span></span></span></span></span></span></span></span></span></span></span>是唯一个在fine-tuning阶段分类层引入的参数，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07153em;">K</span></span></span></span>是分类器数量，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.08125em;">H</span></span></span></span>是Transformer Encoder的隐含层大小。</p>
<h2 id="bert应用到具体nlp任务"> BERT应用到具体NLP任务</h2>
<center><img src="./bert/tasks.png" title="BERT应用在具体的NLP任务" width=60%></center>
<p>(a)<strong>一对句子的分类任务</strong>：多中心的自然语言推断（Multi-centre Natural Language Inference，MNLI）是一个众包蕴含（crowdsource entailment）的分类任务，句子语义等价判断（Quara Question Pairs, QQP）是一个二值分类任务。将两个句子传入BERT，通过<code>[CLS]</code>标志位的输出值（来自Transformer Encoder的输出值）进行句子分类。</p>
<p>(b)<strong>单个句子分类任务</strong>：如句子情感分析（Stanford Sentiment Treebank, SST-2），判断句子语法是否可以接受（The Corpus of Linguistic Acceptability, CoLA）等。输入一个句子，通过<code>[CLS]</code>标志位的输出值进行分类。</p>
<p>(c)<strong>问答任务</strong>：样本是一个语句对（Question，Paragraph），训练的目标是在Paragraph找出答案的起始位置（Start，End）。</p>
<p>(d)<strong>单个句子标注</strong>：如命名实体识别（NER），输入单个句子，然后根据BERT对于每个token的输出<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">T</span></span></span></span>预测该token的类别，是属于Person，Organization，Location，Miscellaneous 还是 Other (非命名实体)。</p>
<h1 id="questions"> Questions</h1>
<p>1、BERT的三个Embedding为什么可以进行相加？</p>
<blockquote>
<p>一串文本可以看作是一些不同频率的时序信号进行叠加，其中每个词的特征可以用叠加波来表示，整个文本序列也可以进一步叠加。哪些是低频信号（如词性？），哪些是高频信号（如语义？），这些都隐藏在Embedding中，也可能已经解耦在不同的维度中。</p>
<p>本质上三个Embedding相加，可以看做是一个特征的融合，BERT可以学习到融合特征的语义信息。</p>
<p>Answer From：https://www.zhihu.com/question/374835153——邱锡鹏老师的回答</p>
</blockquote>
<p>2、使用BERT预训练模型为什么最多只能输入512个词，最多只能两个句子合成一句？</p>
<blockquote>
<p>在BERT中，Token，Position，Segement Embedding都是<strong>通过学习得到</strong>的，Pytorch代码中：</p>
<div><pre><code>self<span>.</span>word_embeddings <span>=</span> Embedding<span>(</span>config<span>.</span>vocab_size<span>,</span> config<span>.</span>hidden_size<span>)</span>
self<span>.</span>position_embeddings <span>=</span> Embedding<span>(</span>config<span>.</span>max_position_embeddings<span>,</span> config<span>.</span>hidden_size<span>)</span>
self<span>.</span>token_type_embeddings <span>=</span> Embedding<span>(</span>config<span>.</span>type_vocab_size<span>,</span> config<span>.</span>hidden_size<span>)</span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><p>Pytorch版的BERT代码地址：https://github.com/xieyufei1993/Bert-Pytorch-Chinese-TextClassification</p>
<p>在BERT config中：</p>
<div><pre><code><span>"max_position_embeddings"</span><span>:</span> <span>512</span>
<span>"type_vocab_size"</span><span>:</span> <span>2</span>
</code></pre>
<div><span>1</span><br><span>2</span><br></div></div><p>因此，在直接使用Google 的BERT预训练模型时，输入最多512个词（还要除掉<code>[CLS]</code>和<code>[SEP]</code>，即实际510个词），最多两个句子合成一句。这之外的词和句子会没有对应的embedding。</p>
<p>BERT本身的训练对于显存有很高的要求，参数量特别大。如果有足够的硬件资源自己重新训练BERT，可以更改 BERT config，设置更大max_position_embeddings 和 type_vocab_size值去满足自己的需求。</p>
<p>Answer From：https://zhuanlan.zhihu.com/p/132554155——超细节的BERT/Transformer知识点</p>
</blockquote>
<p>3、BERT的第一句前加一个<code>[CLS]</code>标志位作用？</p>
<blockquote>
<p>在Transformer Encoder的最后一层该位的输出向量可以作为整个语句的语义表示，从而用于下游的分类任务。与文中已有的其他词相比，这一个无明显语义的信息符号<strong>更能“公平”地融合</strong>文本中各个词的语义信息。</p>
<p>Answer From：https://zhuanlan.zhihu.com/p/132554155——超细节的BERT/Transformer知识点</p>
</blockquote>
<p>4、BERT的非线性来源？</p>
<blockquote>
<p>前馈层的gelu激活函数和self-attention，self-attention是非线性的。</p>
<p>Answer From：https://zhuanlan.zhihu.com/p/132554155——超细节的BERT/Transformer知识点</p>
</blockquote>
<p>5、BERT中，长文本问题解决？</p>
<blockquote>
<p>BERT在进行MLM预训练时，规定了最大的输入长度不超过512个token。对于超过512个token的长文本，无法直接调用开源的pretrained-model进行fine-tuning。在比如阅读理解问题中，输入语句通常远大于512。</p>
<p>(1)<strong>Clipping（截断法）</strong>：挑选其中重要的token输入模型。挑选前TOP N个token和文末TOP K个token，保证<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7667em;vertical-align:-0.0833em;"></span><span style="margin-right:0.10903em;">N</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8193em;vertical-align:-0.136em;"></span><span style="margin-right:0.07153em;">K</span><span style="margin-right:0.2778em;"></span><span>≤</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>510</span></span></span></span>；会丢失信息。</p>
<p>(2)<strong>RNN循环法</strong></p>
<p>(3)<strong>Pooling（池化法）</strong>：借鉴CNN中池化层作用是选取一个区域内最重要的特征值代替整个feature map的特征值（包括最大池化和平均池化），来实现降维功能，整个过程也被称为下采样（down samping）。同理，对于一个1000token的输入文本，得到1000个Embedding，通过Pooling方式获取其中最重要的510个Embedding（另两个是<code>[CLS]</code>,<code>[SEP]</code>）代表整段文本的Embedding。</p>
<p>Answer From：<a href="https://zhuanlan.zhihu.com/p/504204038#:~:text=%E7%94%B1%E4%BA%8E%20BERT%20%E6%9C%80%E5%A4%9A%E5%8F%AA%E8%83%BD%E6%8E%A5%E5%8F%97%20510%20%E4%B8%AAtoken%20%E7%9A%84%E8%BE%93%E5%85%A5%EF%BC%8C%E5%9B%A0%E6%AD%A4%E6%88%91%E4%BB%AC%E9%9C%80%E8%A6%81%E5%B0%86%E9%95%BF%E6%96%87%E6%9C%AC%E5%88%87%E5%89%B2%E6%88%90%E8%8B%A5%E5%B9%B2%E6%AE%B5%E3%80%82%20%E5%81%87%E8%AE%BE%E6%88%91%E4%BB%AC%E6%9C%89%202,%2B%20%E7%AC%AC%202%20%E4%B8%AA%E5%8F%A5%E5%AD%90%E7%9A%84%202%20%E6%AE%B5%EF%BC%89%EF%BC%8C%E5%B9%B6%E6%94%BE%E5%88%B0%E4%B8%80%E4%B8%AA%20batch%20%E7%9A%84%E8%BE%93%E5%85%A5%E4%B8%AD%E5%96%82%E7%BB%99%E6%A8%A1%E5%9E%8B%E3%80%82" target="_blank" rel="noopener noreferrer">一般有如下几种方法——此文主要介绍Pooling</a></p>
</blockquote>
<h2 id="reference"> Reference</h2>
<hr>
<section>
<ol>
<li id="footnote1"><p>BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (Devlin et al., NAACL 2019)   </p>
</li>
<li id="footnote2"><p>理解BERT模型：https://www.jianshu.com/p/46cb208d45c3  </p>
</li>
</ol>
</section>
]]></content:encoded>
    </item>
    <item>
      <title>计算机网络</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/computer%20network/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/computer%20network/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">计算机网络</source>
      <description>整理的关于计算机网络的基础知识</description>
      <content:encoded><![CDATA[<p>参考书目：</p>
<p>《计算机网络——自顶向下方法》（Computer Networking——A Top-Down Approach）</p>
<p>谢希仁《计算机网络》</p>
<p>待更新。。。</p>
]]></content:encoded>
    </item>
    <item>
      <title>Ubuntu16.04配置cuda和cudnn</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/cuda%E5%A4%9A%E7%89%88%E6%9C%AC%E5%AE%89%E8%A3%85/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/cuda%E5%A4%9A%E7%89%88%E6%9C%AC%E5%AE%89%E8%A3%85/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Ubuntu16.04配置cuda和cudnn</source>
      <description>GPU加速必须在cuda9以上才可以完成加速，cuda9以后的版本在Linux系统上只支持Ubuntu16.04和Ubuntu17.04。</description>
      <category>MachineLearning</category>
      <pubDate>Thu, 24 Mar 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<p>GPU加速必须在cuda9以上才可以完成加速，cuda9以后的版本在Linux系统上只支持Ubuntu16.04和Ubuntu17.04。</p>
<h2 id="一、设备要求"> 一、设备要求</h2>
<p>系统：Ubuntu16.04及以上</p>
<p>显卡驱动：NVIDIA系列（384以上）</p>
<p>驱动检查：终端命令<code>nvidia-smi</code></p>
<blockquote>
<p>新安装的Ubuntu系统安装NVIDIA驱动步骤:</p>
<p>a.删除旧驱动：</p>
<p>`sudo apt-get remove --purge nvidia*</p>
<p>b.禁用自带的nouveau nvidia驱动：</p>
<p>看下Nouveau是否已经被禁用</p>
<div><pre><code>lsmod | grep nouveau  
</code></pre>
<div><span>1</span><br></div></div><p>如果已经没有任何显示说明不用禁用了，否则继续下面操作</p>
<div><pre><code>sudo vim /etc/modprobe.d/blacklist-nouveau.conf #创建一个文件（注：按一下i键，表示现在进行内容插入）
</code></pre>
<div><span>1</span><br></div></div><p>并添加如下内容：</p>
<div><pre><code>blacklist nouveau
options nouveau modeset=0 
</code></pre>
<div><span>1</span><br><span>2</span><br></div></div></blockquote>
<h2 id="二、安装cuda9-0"> 二、安装cuda9.0</h2>
<p><a href="https://developer.nvidia.com/cuda-90-download-archive?target_os=Linux&amp;target_arch=x86_64&amp;target_distro=Ubuntu&amp;target_version=1604&amp;target_type=runfilelocal" target="_blank" rel="noopener noreferrer">官网下载</a>。有对应的deb和.run文件，有具体的安装命令。</p>
<p>推荐.run文件，因为deb文件安装，可能会修改已安装好的驱动和。</p>
<p>进入下载目录，给文件添加运行权限：</p>
<div><pre><code>chmod +x ./cuda_9.0.176_384.81_linux.run 
</code></pre>
<div><span>1</span><br></div></div><p>运行安装</p>
<div><pre><code>sudo ./cuda_9.0.176_384.81_linux.run 
</code></pre>
<div><span>1</span><br></div></div><p>启动安装程序，一直按空格到最后（可以选择Ctrl+c跳过），不用担心，到99%的时候，输入accept接受条款</p>
<p>注意：</p>
<ul>
<li>第一个提醒你是否安装驱动时，选“n”，其余都“y”，安装路径选择默认的路径。</li>
<li>如果需要安装多版本CUDA，在安装到选择symtomic link to <code>/urs/local/cuda</code>时候，选择'n'，其它都是&quot;y&quot;。</li>
</ul>
<p><strong>以上为两种cuda的安装方法，不过并没有结束，安装完毕后必须添加环境</strong></p>
<div><pre><code>gedit ~/.bashrc 
</code></pre>
<div><span>1</span><br></div></div><div><pre><code># 在文件末尾添加环境变量
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-9.0/lib64
export PATH=$PATH:/usr/local/cuda-9.0/bin
export CUDA_HOME=$CUDA_HOME:/usr/local/cuda-9.0
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br></div></div><p>添加完后，终端命令：<code>source ～/.bashrc</code></p>
<p>终端命令查看是否安装成功：<code>nvcc -V</code>，若出现nvcc未安装，</p>
<p>执行了 sudo apt-get install nvidia-cuda-toolkit 命令，nvcc命令可以正常执行，但是运行nvcc --version发现版本为5.5，与原来安装的CUDA 10.0不匹配，这将会导致一系列不兼容问题（比如在python中使用sk-cuda库就报错，因为nvcc版本不一致），遂通过如下步骤解决：</p>
<p>1、使用sudo apt-get autoremove nvidia-cuda-toolkit 卸载5.5版本</p>
<p>2、查看/usr/local/cuda/bin下是否有nvcc可执行程序，如果没有说明cuda没有正常安装，需要重新安装。</p>
<p>3、如果有nvcc可执行程序，切换到管理员权限，重新执行上面步骤，添加环境。</p>
<h2 id="三、安装cudnn7-3-1"> 三、安装cudnn7.3.1</h2>
<p>去<a href="https://developer.nvidia.com/rdp/cudnn-archive" target="_blank" rel="noopener noreferrer">官网下载</a>，此处下载的是<code>cuDNN v7.3.1 Library for Linux</code>版本。是一个.tgz文件。</p>
<p>下载完直接解压，解压会出现一个cuda文件夹，里面有两个文件include 和lib64，把里面的文件<code>copy到/usr/local/cuda-9.0</code>里面相应的目录里，具体命令如下：</p>
<div><pre><code>sudo cp cuda/include/cudnn.h /usr/local/cuda-9.0/include
sudo cp cuda/lib64/libcudnn* /usr/local/cuda-9.0/lib64
sudo chmod a+r /usr/local/cuda-9.0/include/cudnn.h /usr/local/cuda-9.0/lib64/libcudnn*
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><p>查看cudnn版本，检查是否安装好：</p>
<div><pre><code>cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2 
</code></pre>
<div><span>1</span><br></div></div><h2 id="四、卸载"> 四、卸载</h2>
<p>（1）卸载CUDA</p>
<div><pre><code>sudo /usr/local/cuda-9.0/bin/uninstall_cuda_9.0.pl 
</code></pre>
<div><span>1</span><br></div></div><p>（2）卸载CUDNN</p>
<p>删除原来的cudnn文件：</p>
<div><pre><code>sudo rm -rf /usr/local/cuda-9.0/lib64/libcudnn*
sudo rm -rf /usr/local/cuda-9.0/include/cudnn.h
</code></pre>
<div><span>1</span><br><span>2</span><br></div></div><p><a href="https://zhuanlan.zhihu.com/p/58114055" target="_blank" rel="noopener noreferrer">Ubuntu16.04下cuda9.0+cudnn7.0安装参考教程1</a></p>
<p><a href="https://www.jianshu.com/p/6a6fbce9073f" target="_blank" rel="noopener noreferrer">Ubuntu 多版本Cuda(8.0,9.0)以及CuDnn安装参考教程2</a></p>
<hr>
<h2 id="pytorch-gpu环境配置"> pytorch_gpu环境配置</h2>
<p>前提是已经配置好cuda9.0和cudnn7.0</p>
<p>在Anaconda环境下，新建一个pytorch-gpu环境，安装pytorch。</p>
<p>一、创建新的虚拟环境</p>
<div><pre><code>conda create -n pytorch_gpu python=3.7
conda activate pytorch_gpu /source activate pytorch_gpu #激动虚拟环境
</code></pre>
<div><span>1</span><br><span>2</span><br></div></div><p>二、安装pytorch GPU版本</p>
<p>此处是官网直接下载安装，具体考虑下载速度，可离线安装</p>
<div><pre><code>conda install pytorch=1.7.1 torchvision torchaudio cudatoolkit=9.2 -c pytorch
</code></pre>
<div><span>1</span><br></div></div><p>下载速度过慢，选用国内源：</p>
<div><pre><code>##win
conda install pytorch=1.7.1 torchvision torchaudio cudatoolkit=9.2 -c https://mirrors.bfsu.edu.cn/anaconda/cloud/pytorch/win-64/

##linux
conda install pytorch=1.7.1 torchvision torchaudio cudatoolkit=9.2 -c https://mirrors.bfsu.edu.cn/anaconda/cloud/pytorch/linux-64/
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br></div></div><p><code>torchvision</code>和<code>torchaudio</code>跟随安装的python版本和pytorch版本自动对应。</p>
<p>pytorch_gpu使用<strong>cudatoolkit（是cuda的子集，里面的东西足够pytorch-gpu版本使用</strong>），相当于虚拟环境内置的cuda，不需要系统cuda。</p>
<hr>
<h2 id="tensorflow-gpu-keras环境配置"> tensorflow_gpu_keras环境配置</h2>
<p>Keras其实就是TensorFlow和Keras的接口（Keras作为前端，TensorFlow或theano作为后端），它也很灵活，且比较容易学。可以把keras看作为tensorflow封装后的一个API。</p>
<p>目前Keras是tensorflow官方的API。</p>
<h3 id="查看已经安装的python版本"> 查看已经安装的Python版本</h3>
<div><pre><code>ls | grep python
</code></pre>
<div><span>1</span><br></div></div><h3 id="将pip服务器更换为国内的镜像服务器"> 将pip服务器更换为国内的镜像服务器</h3>
<div><pre><code># 如果不存在此文件夹，则创建之
mkdir ~/.pip
vim ~/.pip/pip.conf
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><p>内容如下</p>
<div><pre><code>[global]
index-url=http://mirrors.aliyun.com/pypi/simple/
[install]
trusted-host=mirrors.aliyun.com
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br></div></div><h3 id="tensorflow-gpu和keras安装"> tensorflow-gpu和keras安装</h3>
<h4 id="一、新建虚拟环境"> 一、新建虚拟环境</h4>
<div><pre><code>conda create -n tensorflow_gpu python=3.6 (ipykernel) #选择加如jupyter notebook内核
</code></pre>
<div><span>1</span><br></div></div><p>找到环境支持的对应的版本：</p>
<p><img src="https://img-blog.csdnimg.cn/20190819221921944.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L1RZVVRfeGlhb21pbmc=,size_16,color_FFFFFF,t_70" alt="img" loading="lazy"></p>
<h4 id="二、安装tensorflow-gpu"> 二、安装tensorflow-gpu</h4>
<p>终端命令：</p>
<div><pre><code>pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.6.0-cp36-cp36m-linux_x86_64.whl
</code></pre>
<div><span>1</span><br></div></div><div><pre><code>#激活虚拟环境后，通过终端直接安装
pip install tensorflow-gpu==1.14.0
</code></pre>
<div><span>1</span><br><span>2</span><br></div></div><h4 id="三、测试tensorflow-gpu是否安装成功"> 三、测试tensorflow-gpu是否安装成功</h4>
<p>测试代码如下：</p>
<div><pre><code>import tensorflow as tf
a = tf.constant([1.,2.,3.,4.,5.,6.], shape=[2,3], name=&#39;a&#39;)
b = tf.constant([1.,2.,3.,4.,5.,6.], shape=[3,2], name=&#39;b&#39;)
c = tf.matmul(a,b)

with tf.Session(config= tf.ConfigProto(log_device_placement=True)) as sess:
    print(sess.run(c))
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br></div></div><p>成功检测到GPU，表明安装成功，且可以调用GPU运行。</p>
<p>GPU使用率：terminal下run：<code>nvidia-smi -q -g 0 -d UTILIZATION -l</code>
or run：<code>watch -n 2 nvidia-smi</code></p>
<p>注意可能会出现tensorflow-gpu版本和cuda，cudnn版本的不匹配问题，需要卸载</p>
<p><a href="https://blog.csdn.net/linhai1028/article/details/79525507" target="_blank" rel="noopener noreferrer">ubuntu下tensorflow-gpu环境搭建</a></p>
<h4 id="四、查看tensorflow-gpu安装版本"> 四、查看tensorflow-gpu安装版本</h4>
<div><pre><code>在python环境中输入：

import tensorflow as tf

tf.__version__     (查看版本）

tf.__path__        （查看路径）
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br></div></div><h4 id="五、卸载命令"> 五、卸载命令</h4>
<div><pre><code>sudo pip uninstall tensorflow
sudo pip3 uninstall tensorflow
</code></pre>
<div><span>1</span><br><span>2</span><br></div></div><h4 id="六、安装keras"> 六、安装Keras</h4>
<div><pre><code>pip install keras -U --pre
</code></pre>
<div><span>1</span><br></div></div><h4 id="七、验证安装"> 七、验证安装</h4>
<div><pre><code>进入python命令环境

import keras
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><div><pre><code>Keras测试：

Keras中mnist数据集测试 下载Keras开发包

conda install git

git clone https://github.com/fchollet/keras.git

cd keras/examples/

python mnist_mlp.py

</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br></div></div><p>在《Tensorflow技术解析与实战》一书中提到，keras安装后需要选择依赖的后端，在~/.Keras/keras.json下修改最后一行backend对应的值，（默认后端是tensorflow，不用修改）也就是：</p>
<div><pre><code>｛

&quot;image_dim_ordering&quot;:&quot;tf&quot;,

&quot;epsilon&quot;:le-07,

&quot;floatx&quot;:&quot;float32&quot;

&quot;backend&quot;: &quot;tensorflow&quot;

｝
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br></div></div><h4 id="八、出现问题"> 八、出现问题</h4>
<p>安装keras出现：<code>ImportError: cannot import name 'tf_utils'</code></p>
<div><pre><code>&gt;&gt;&gt; import tensorflow
&gt;&gt;&gt; import keras
Using TensorFlow backend.
Traceback (most recent call last):
  File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
  File &quot;/home/reyhan/project/.virtualenvs/keras_tf/lib/python3.6/site-packages/keras/__init__.py&quot;, line 3, in &lt;module&gt;
from . import utils
  File &quot;/home/reyhan/project/.virtualenvs/keras_tf/lib/python3.6/site-packages/keras/utils/__init__.py&quot;, line 6, in &lt;module&gt;
from . import conv_utils
  File &quot;/home/reyhan/project/.virtualenvs/keras_tf/lib/python3.6/site-packages/keras/utils/conv_utils.py&quot;, line 9, in &lt;module&gt;
from .. import backend as K
  File &quot;/home/reyhan/project/.virtualenvs/keras_tf/lib/python3.6/site-packages/keras/backend/__init__.py&quot;, line 1, in &lt;module&gt;
from .load_backend import epsilon
  File &quot;/home/reyhan/project/.virtualenvs/keras_tf/lib/python3.6/site-packages/keras/backend/load_backend.py&quot;, line 90, in &lt;module&gt;
from .tensorflow_backend import *
  File &quot;/home/reyhan/project/.virtualenvs/keras_tf/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py&quot;, line 13, in &lt;module&gt;
from tensorflow.python.keras.utils import tf_utils
ImportError: cannot import name &#39;tf_utils&#39;
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br><span>11</span><br><span>12</span><br><span>13</span><br><span>14</span><br><span>15</span><br><span>16</span><br><span>17</span><br><span>18</span><br></div></div><p>安装的keras版本是2.3.1的，卸载安装keras2.1.5版本，解决该问题，首先卸载现有keras版本：</p>
<div><pre><code>pip uninstall keras
</code></pre>
<div><span>1</span><br></div></div><p>安装keras2.1.5版本：</p>
<div><pre><code>pip install keras==2.1.5
</code></pre>
<div><span>1</span><br></div></div><h2 id="cuda多版本安装"> cuda多版本安装</h2>
<h3 id="下载"> 下载</h3>
<p>cuda9.0安装：</p>
<p>https://developer.nvidia.com/cuda-90-download-archive?target_os=Linux&amp;target_arch=x86_64&amp;target_distro=Ubuntu&amp;target_version=1604&amp;target_type=runfilelocal</p>
<p>cuda历史版本下载：</p>
<p>https://developer.nvidia.com/cuda-toolkit-archive</p>
<p>cudnn7.6.5安装：</p>
<p>https://developer.nvidia.com/rdp/cudnn-archive</p>
<p>注意版本对应</p>
<h3 id="环境配置"> 环境配置</h3>
<p>cuda安装完成后，配置cuda环境：</p>
<p>在<code>～/.bashrc</code>文件中修改或者添加如下：</p>
<div><pre><code>vim ~/.bashrc

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64
export PATH=$PATH:/usr/local/cuda/bin
export CUDA_HOME=$CUDA_HOME:/usr/local/cuda

source ~/.bashrc
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br></div></div><p>若之前安装过别的版本，上述文件修改为cuda即可，使用cuda这个软链接选择指定cuda版本。</p>
<h3 id="多版本切换"> 多版本切换</h3>
<p>在/usr/locala目录下：</p>
<div><pre><code>#切换cuda版本
rm -rf /usr/local/cuda  #删除之前创建的软链接
sudo ln -s /usr/local/cuda-9.2/ /usr/local/cuda   #添加新版本了连接
nvcc -V  #查看当前cuda版本
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br></div></div><p>参考：https://blog.csdn.net/weixin_42416791/article/details/115720595</p>
<h3 id="问题"> 问题</h3>
<p>nvcc –V查看的cuda版本不是软链接的cuda对应的版本问题解决</p>
<p>修改<code>/etc/profile</code>，在末尾加上：</p>
<div><pre><code>vim /etc/profile

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64$LD_LIBRARY_PATH
export CUDA_HOME=/usr/local/cuda

source /etc/profile
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br></div></div><p>然后更改PATH内顺序：</p>
<div><pre><code>root@jiang-X299:/home/jiang/桌面# export PATH=/usr/local/cuda/bin:$PATH
root@jiang-X299:/home/jiang/桌面# echo $PATH
/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/cuda-9.0/bin:/home/jiang/anaconda3/bin
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><p>参考1：https://blog.csdn.net/weixin_42416791/article/details/115720595</p>
<p>参考2：https://www.jianshu.com/p/6a6fbce9073f</p>
<h3 id="版本查询"> 版本查询</h3>
<p>安装完成cuda后 还需要安装对应版本的cudnn</p>
<p>使用 PyTorch 查看 CUDA 和 cuDNN 版本：</p>
<div><pre><code>import torch
print(torch.__version__)
print(torch.version.cuda)
print(torch.backends.cudnn.version())
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br></div></div><p>查看cuda版本方法：</p>
<div><pre><code>nvcc -V
或
nvcc --version
或
cat /usr/local/cuda/version.txt
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br></div></div><p>查看cudnn版本：</p>
<div><pre><code>cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
</code></pre>
<div><span>1</span><br></div></div><div><pre><code>版本：7.6.5

#define CUDNN_MAJOR 7
#define CUDNN_MINOR 6
#define CUDNN_PATCHLEVEL 5
--
#define CUDNN_VERSION (CUDNN_MAJOR * 1000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL)
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br></div></div><h2 id="pytorch安装过慢问题"> pytorch安装过慢问题</h2>
<p>直接利用<code>conda/pip install XXX</code> 从pytorch官网安装代码速度太慢，会出现进度中断情况。通常的做法是添加清华镜像的下载路径。</p>
<h3 id="conda镜像源修改"> conda镜像源修改</h3>
<div><pre><code>vim ~/.condarc
</code></pre>
<div><span>1</span><br></div></div><p>添加如下内容：</p>
<div><pre><code>channels:
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/Paddle/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/fastai/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/
show_channel_urls: true
ssl_verify: false
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br></div></div><p>然后再去pytorch官网复制下载代码，删掉后面的-c pytorch,因为它表示从官网进行下载，不从清华镜像源下：</p>
<div><pre><code>conda install pytorch torchvision cudatoolkit=9.0 -c pytorch
修改为：
conda install pytorch torchvision cudatoolkit=9.0
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><h3 id="pip镜像源修改"> pip镜像源修改</h3>
<div><pre><code>sudo gedit ~/.pip/pip.conf
</code></pre>
<div><span>1</span><br></div></div><p>添加如下内容：</p>
<div><pre><code>[global]
#index-url=http://mirrors.aliyun.com/pypi/simple/
#index-url = https://pypi.tuna.tsinghua.edu.cn/simple/
#index-url=https://pypi.python.org/simple/
index-url=https://pypi.douban.com/simple/
[install]
#trusted-host=mirrors.aliyun.com
#trusted-host=pypi.tuna.tsinghua.edu.cn
#trusted-host=pypi.python.org
trusted-hosr=pypi.douban.com
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br></div></div><p>若没有<code>pip.conf</code>文件，创建如下：</p>
<div><pre><code>cd ~/.pip
touch pip.conf

若没有目录：
mkdir ~/.pip
cd ~/.pip
touch pip.conf
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br></div></div><h3 id="离线安装"> 离线安装</h3>
<p>但是有时候更换国内的源，下载仍会很慢，会中断，考虑去镜像源网站直接下载，然后离线安装，方法如下：</p>
<p>此处以ubuntu16.04系统下为例：</p>
<p>下载安装包：</p>
<div><pre><code>pytorch-1.7.1-py3.7_cuda9.2.148_cudnn7.6.3_0.tar.bz2
torchaudio-0.7.2-py37.tar.bz2
torchvision-0.8.2-py37_cu92.tar.bz2
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><p>激活创建好的虚拟环境（pytorch），在该虚拟环境下，切换到下载文件的位置，执行命令安装：</p>
<div><pre><code>(pytorch) $ conda install --offline pytorch-1.7.1-py3.7_cuda9.2.148_cudnn7.6.3_0.tar.bz2
(pytorch) $ conda install --offline torchaudio-0.7.2-py37.tar.bz2
(pytorch) $ conda install --offline torchvision-0.8.2-py37_cu92.tar.bz2
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br></div></div><p>下载对应版本的<code>pytorch、torchvision、cudatoolkit等</code>，然后离线安装。</p>
<p>离线安装注意切换到对应的虚拟环境下进行，cd命令把当前目录改到下载的离线文件所在目录。</p>
<p>最好离线安装安装包的文件，对于一些依赖包，可以先在线安装好，最后离线安装如pytorch、torchvision、cudatoolkit等下载很慢的安装包。</p>
]]></content:encoded>
      <enclosure url="https://img-blog.csdnimg.cn/20190819221921944.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L1RZVVRfeGlhb21pbmc=,size_16,color_FFFFFF,t_70" type="image/"/>
    </item>
    <item>
      <title>生成模型</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/NLP/generative_model/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/NLP/generative_model/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">生成模型</source>
      <description>生成模型原理分析</description>
      <category>自然语言处理</category>
      <pubDate>Tue, 15 Nov 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h1 id="introduction"> Introduction</h1>
<p><strong>Generative models</strong> are one of the most promising approaches to analyze and understand the treasure trove(宝库) of data, either in the physical world of atoms or the digital world of bits.</p>
<p>To train a generative model we first collect a large amount of data in some domain(e.g. think millions of images, sentences, or sounds, etc.) and then train a model to generate data like it.</p>
<p>The neural networks, which have a number of parameters significantly smaller than the amount of train data, can be used as generative models to discover and efficiently internalize(内在化) the essence(本质) of the data in order to generate it.</p>
<p>Generative models hold the potential to automatically learn the natural features of a dataset, whether categories or dimensions or something else entirely[^OpenAi].</p>
<h1 id="references"> References</h1>
]]></content:encoded>
    </item>
    <item>
      <title>人工智能的数学基础知识</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/mathematics/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/mathematics/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">人工智能的数学基础知识</source>
      <description>整理的关于在人工智能领域中用到的数学知识。</description>
      <content:encoded><![CDATA[<p>计算机科学基本上是应用数学的一个“跑偏的”分支，对于计算机科学，数学中最相关的领域是“离散数学”，其中的“离散”与“连续”相对立，大致上指的是应用数学中那些有趣的主题，而不是微积分之类的。</p>
<p>一个比较现实的目标是了解：逻辑、排列组合、概率论、集合论、图论以及密码学相关的一些数论知识。此外，线性代数在机器学习和计算图形学中具有举足轻重的作用，也同样值得学习。</p>
]]></content:encoded>
    </item>
    <item>
      <title>计算机网络概述</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/computer%20network/%E8%AE%A1%E7%AE%97%E6%9C%BA%E7%BD%91%E7%BB%9C%E7%AC%AC1%E7%AB%A0%EF%BC%88%E6%A6%82%E8%BF%B0%EF%BC%89/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/computer%20network/%E8%AE%A1%E7%AE%97%E6%9C%BA%E7%BD%91%E7%BB%9C%E7%AC%AC1%E7%AB%A0%EF%BC%88%E6%A6%82%E8%BF%B0%EF%BC%89/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">计算机网络概述</source>
      <description>关于计算机网络的基本概述，包括计算机网络的定义、分类、性能指标和体系结构等。</description>
      <category>ComputerScience</category>
      <pubDate>Thu, 03 Mar 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="图示说明"> 图示说明</h2>
<p><img src="./计算机网络第1章（概述）.assets/20201016103344.png" title="image-20201007143815954" width=8% />代表着主机</p>
<p><img src="./计算机网络第1章（概述）.assets/20201016103558.png" title="image-20201007144130208" width=8% /> 代表服务器</p>
<p><img src="./计算机网络第1章（概述）.assets/20201016103603.png"  title="image-20201007143904153"  width=8% /> 代表着路由器</p>
<p><img src="./计算机网络第1章（概述）.assets/20201016103608.png" title="image-20201007143936128" width=8% />代表着网络</p>
<h2 id="因特网概述"> 因特网概述</h2>
<h3 id="网络、互连网-互联网-和因特网"> 网络、互连网（互联网）和因特网</h3>
<p><font size=5><strong>网络</strong></font>：网络（Network）由若干**结点（Node）<strong>和连接这些结点的</strong>链路（Link）**组成。</p>
<center><img src="./计算机网络第1章（概述）.assets/20201016103711.png" title="image-20201006163406232" width=30% /></center>
<p><font size=5><strong>互连网（互联网）</strong></font>：多个网络通过路由器互连起来，这样就构成了一个覆盖范围更大的网络，即互连网（互联网）。因此，互联网又称为“网络的网络（Network of Networks）”。</p>
<center><img src="./计算机网络第1章（概述）.assets/20201016103716.png" title="image-20201006163857236" width=50% /></center>
<p><font size=5><strong>因特网</strong></font>：因特网（Internet）是世界上最大的互连网络（用户数以亿计，互连的网络数以百万计）。</p>
<center><img src="./计算机网络第1章（概述）.assets/20201016103720.png" title="image-20201006164114523" width=50% /></center>
<blockquote>
<p><strong>internet与Internet的区别</strong></p>
<ul>
<li><strong>internet(互联网或互连网)<strong>是一个通用名词，它泛指</strong>多个计算机网络互连而成的网络</strong>。在这些网络之间的通信协议可以是任意的。</li>
<li><strong>Internet（因特网）则是一个专用名词</strong>，它指<strong>当前全球最大的、开放的、由众多网络互连而成的特定计算机网络</strong>，它采用TCP/IP协议族作为通信的规则，其前身是美国的ARPANET。</li>
</ul>
<p><strong>任意把几个计算机网络互连起来（不管采用什么协议），并能够相互通信，这样构成的是一个互连网(internet) ，而不是互联网(Internet)。</strong></p>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016103731.png" width=80%></center>
<h3 id="因特网发展的三个阶段"> 因特网发展的三个阶段</h3>
<center><img src="./计算机网络第1章（概述）.assets/20201016103739.png" width=90%></center>
<p><strong>因特网服务提供者<code>ISP</code>(<code>I</code>nternet <code>S</code>ervice <code>P</code>rovider)</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016103744.png" title="image-20201006165117756" width=70% /> </center>
<blockquote>
<p>普通用户是如何接入到因特网的呢？</p>
<p>答：<strong>通过ISP接入因特网</strong></p>
<p>ISP可以从因特网管理机构申请到成块的IP地址，同时拥有通信线路以及路由器等联网设备。任何机构和个人只需缴纳费用，就可从<strong>ISP的得到所需要的IP地址</strong>。</p>
<p><strong>因为因特网上的主机都必须有IP地址才能进行通信，这样就可以通过该ISP接入到因特网</strong></p>
</blockquote>
<p><strong>中国的三大<code>ISP</code>：中国电信，中国联通和中国移动</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016103801.png" title="image-20201006165738879" width=70% /></center>
<p><strong>基于ISP的三层结构的因特网</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016103810.png" title="image-20201006170140504" width=80% /></center>
<blockquote>
<p>一旦某个用户能够接入到因特网，那么他也可以成为一个ISP，所需要做的就是购买一些如调制解调器或路由器这样的设备，让其他用户可以和他相连。</p>
</blockquote>
<h3 id="因特网的组成"> 因特网的组成</h3>
<ul>
<li>
<p>边缘部分</p>
<p>由所有连接在因特网上的<strong>主机</strong>组成（台式电脑，大型服务器，笔记本电脑，平板，智能手机等）。这部分是<strong>用户直接使用</strong>的，用来进行<strong>通信</strong>（传送数据、音频或视频）和<strong>资源共享</strong>。</p>
</li>
<li>
<p>核心部分</p>
<p>由<strong>大量网络</strong>和连接这些网络的<strong>路由器</strong>组成。这部分是<strong>为边缘部分提供服务</strong>的（提供连通性和交换）。</p>
</li>
</ul>
<center><img src="./计算机网络第1章（概述）.assets/20201016103822.png" title="image-20201006180725282" width=80% /></center>
<blockquote>
<p>路由器是一种专用计算机，但我们不称它为主机，路由器是实现分组交换的关键构建，其任务是转发收到的分组，这是网络核心最重要的部分。</p>
<p>处在互联网边缘的部分就是连接在互联网上的所有的主机。这些主机又称为<strong>端系统 (end system)</strong>。</p>
<p><strong>端系统在功能上可能有很大的差别：</strong></p>
<ol>
<li>
<p>小的端系统可以是一台普通个人电脑，具有上网功能的智能手机，甚至是一个很小的网络摄像头。</p>
</li>
<li>
<p>大的端系统则可以是一台非常昂贵的大型计算机。</p>
</li>
<li>
<p>端系统的拥有者可以是个人，也可以是单位（如学校、企业、政府机关等），当然也可以是某个ISP。</p>
</li>
</ol>
</blockquote>
<h3 id="补充"> 补充</h3>
<p><strong>端系统之间通信的含义</strong></p>
<p>“主机 A 和主机 B 进行通信”实际上是指：“运行在主机 A 上的某个程序和运行在主机 B 上的另一个程序进行通信”。<strong>即“主机</strong> <strong>A</strong> <strong>的某个进程和主机</strong> <strong>B</strong> <strong>上的另一个进程进行通信”。简称为“计算机之间通信”。</strong></p>
<p>端系统之间的通信方式通常可划分为两大类：</p>
<center><img src="./计算机网络第1章（概述）.assets/20201016103829.png" width=80%></center>
<p><strong>客户-服务器方式：</strong></p>
<ul>
<li>客户 (client) 和服务器 (server) 都是指通信中所涉及的两个应用进程。</li>
<li>客户 - 服务器方式所描述的是进程之间服务和被服务的关系。</li>
<li>客户是服务的请求方，服务器是服务的提供方。</li>
</ul>
<blockquote>
<p><strong>服务请求方和服务提供方都要使用网络核心部分所提供的服务。</strong></p>
</blockquote>
<p><strong>对等连接方式：</strong></p>
<ul>
<li><strong>对等连接</strong> (peer-to-peer，简写为 <strong>P2P</strong> ) 是指两个主机在通信时并不区分哪一个是服务请求方还是服务提供方。</li>
<li>只要两个主机都运行了对等连接软件 ( P2P 软件) ，它们就可以进行<strong>平等的、对等连接通信</strong>。</li>
<li>双方都可以下载对方已经存储在硬盘中的共享文档。</li>
</ul>
<hr>
<h2 id="三种交换方式"> 三种交换方式</h2>
<p>网络核心部分是互联网中最复杂的部分。</p>
<p>网络中的核心部分要向网络边缘中的大量主机提供连通性，使边缘部分中的任何一个主机都能够向其他主机通信（即传送或接收各种形式的数据）。</p>
<p>在网络核心部分起特殊作用的是<strong>路由器</strong>(router)。</p>
<p><strong>路由器</strong>是实现<strong>分组交换</strong> (packet switching) 的关键构件，其任务是<strong>转发</strong>收到的分组，这是网络核心部分最重要的功能。</p>
<h3 id="电路交换-circuit-switching"> 电路交换（Circuit Switching）</h3>
<center><img src="./计算机网络第1章（概述）.assets/20201016103836.png" title="image-20201006182240000" width=90% /></center>
<blockquote>
<p>传统两两相连的方式，当电话数量很多时，电话线也很多，就很不方便</p>
<p>所以要使得每一部电话能够很方便地和另一部电话进行通信，就应该使用一个<strong>中间设备</strong>将这些电话连接起来，这个中间设备就是<strong>电话交换机</strong></p>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016103840.png" title="image-20201006182634249" width=40% /></center>
<ul>
<li>
<p>电话交换机接通电话线的方式称为电路交换；</p>
</li>
<li>
<p>从通信资源的分配角度来看，交换（Switching）就是按照某种方式动态地分配传输线路的资源；</p>
</li>
<li>
<p>电路交换的三个步骤：</p>
<p>1、建立连接（分配通信资源）</p>
<p>2、通话（一直占用通信资源）</p>
<p>3、释放连接（归还通信资源）</p>
</li>
</ul>
<center><img src="./计算机网络第1章（概述）.assets/20201016103845.png" title="image-20201006183020317" width=90% /></center>
<blockquote>
<p>当使用电路交换来传送计算机数据时，其线路的传输效率往往很低。</p>
<p>这是因为计算机数据是突发式地出现在传输线路上的。</p>
<p>所以计算机通常采用的是<strong>分组交换</strong>，而不是线路交换</p>
</blockquote>
<h3 id="分组交换-packet-switching"> 分组交换（Packet Switching）</h3>
<center><img src="./计算机网络第1章（概述）.assets/20201016103852.png" title="image-20201006183312843" width=90% /></center>
<blockquote>
<p>通常我们把表示<strong>该消息的整块数据</strong>成为一个<strong>报文</strong>。</p>
<p>在发送报文之前，先把较长的报文划分成一个个更小的<strong>等长数据段</strong>，在每一个数据段前面。加上一些由必<strong>要的控制信息组成的首部</strong>后，就构成一个分组，也可简称为“包”，相应地，首部也可称为“包头”。</p>
<p>首部包含了<strong>分组的目的地址</strong></p>
<p>分组从源主机到目的主机，可走不同的路径。</p>
</blockquote>
<p>发送方</p>
<ul>
<li>构造分组</li>
<li>发送分组</li>
</ul>
<p>路由器</p>
<ul>
<li>缓存分组</li>
<li>转发分组</li>
<li>简称为“分组转发”</li>
</ul>
<blockquote>
<p>在路由器中的输入和输出端口之间没有直接连线。</p>
<p>路由器处理分组的过程是：</p>
<ol>
<li>把收到的分组先<strong>放入缓存（暂时存储）；</strong></li>
<li><strong>查找转发表</strong>，找出到某个目的地址应从哪个端口转发；</li>
<li>把分组送到适当的端口<strong>转发</strong>出去。</li>
</ol>
</blockquote>
<p>接收方</p>
<ul>
<li>接收分组</li>
<li>还原报文</li>
</ul>
<h3 id="报文交换-message-switching"> 报文交换（Message Switching）</h3>
<p>报文交换中的交换结点也采用存储转发方式，但报文交换对报文的大小没有限制，这就要求交换结点需要较大的缓存空间。报文交换主要用于早期的电报通信网，现在较少使用，<strong>通常被较先进的分组交换方式所取代</strong>。</p>
<h3 id="三种交换方式的对比"> 三种交换方式的对比</h3>
<blockquote>
<p>假设A，B，C，D是分组传输路径所要经过的4个结点交换机，纵坐标为时间</p>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016103900.png" title="image-20201006184451671" width=90% /></center>
<p>分析：</p>
<p>电路交换：</p>
<ul>
<li>
<p>通信之前首先要建立连接；连接建立好之后，就可以使用已建立好的连接进行数据传送；数据传送后，需释放连接，以归还之前建立连接所占用的通信线路资源。</p>
</li>
<li>
<p>一旦建立连接，中间的各结点交换机就是直通形式的，比特流可以直达终点；</p>
</li>
</ul>
<p>报文交换：</p>
<ul>
<li>可以随时发送报文，而不需要事先建立连接；整个报文先传送到相邻结点交换机，全部存储下来后进行查表转发，转发到下一个结点交换机。</li>
<li>整个报文需要在各结点交换机上进行存储转发，由于不限制报文大小，因此需要各结点交换机都具有较大的缓存空间。</li>
</ul>
<p>分组交换：</p>
<ul>
<li>可以随时发送分组，而不需要事先建立连接。构成原始报文的一个个分组，依次在各结点交换机上存储转发。各结点交换机在发送分组的同时，还缓存接收到的分组。</li>
<li>构成原始报文的一个个分组，在各结点交换机上进行存储转发，相比报文交换，减少了转发时延，还可以避免过长的报文长时间占用链路，同时也有利于进行差错控制。</li>
</ul>
<center><img src="./计算机网络第1章（概述）.assets/20201016103949.png" width=80%></center>
<hr>
<h2 id="计算机网络的定义和分类"> 计算机网络的定义和分类</h2>
<h3 id="定义"> 定义</h3>
<ul>
<li>计算机网络的精确定义并未统一</li>
<li>计算机网络的最简单的定义是：一些<strong>互</strong>相<strong>连</strong>接的、<strong>自治</strong>的计算机的<strong>集合</strong>。
<ul>
<li>互连：是指计算机之间可以通过有线或无线的方式进行数据通信；</li>
<li>自治：是指独立的计算机，他有自己的硬件和软件，可以单独运行使用；</li>
<li>集合：是指至少需要两台计算机；</li>
</ul>
</li>
<li>计算机网络的较好的定义是：计算机网络主要是由一些<strong>通用的，可编程的硬件（一定包含有中央处理机CPU）互连</strong>而成的，而这些硬件并非专门用来实现某一特定目的（例如，传送数据或视频信号）。这些可编程的硬件能够用来<strong>传送多种不同类型的数据</strong>，并能<strong>支持广泛的和日益增长的应用</strong>。
<ul>
<li>计算机网络所连接的硬件，并不限于一般的计算机，而是包括了智能手机等智能硬件。</li>
<li>计算机网络并非专门用来传送数据，而是能够支持很多种的应用（包括今后可能出现的各种应用）。</li>
</ul>
</li>
</ul>
<h3 id="分类"> 分类</h3>
<p><strong>按交换技术分类：</strong></p>
<ul>
<li>电路交换网络</li>
<li>报文交换网络</li>
<li>分组交换网络</li>
</ul>
<p><strong>按使用者分类：</strong></p>
<ul>
<li>公用网</li>
<li>专用网</li>
</ul>
<p><strong>按传输介质分类：</strong></p>
<ul>
<li>有线网络</li>
<li>无线网络</li>
</ul>
<p><strong>按覆盖范围分类：</strong></p>
<ul>
<li>广域网WAN（Wide Area Network）</li>
</ul>
<p>作用范围通常为几十到几千公里，因而有时也称为远程网（long haul network）。广域网是互联网的核心部分，其任务是通过长距离（例如，跨越不同的国家）运送主机所发送的数据。</p>
<ul>
<li>城域网MAN</li>
</ul>
<p>作用范围一般是一个城市，可跨越几个街区甚至整个城市</p>
<ul>
<li>局域网LAN</li>
</ul>
<p>一般用微型计算机或工作站通过高速通信线路相连（速率通常在 10 Mbit/s 以上），但地理上范围较小（1 km 左右）</p>
<ul>
<li>个域网PAN</li>
</ul>
<p>就是在个人工作的地方把个人使用的电子设备用无线技术连接起来的网络。</p>
<p><strong>按拓扑结构分类：</strong></p>
<ul>
<li>总线型网络</li>
</ul>
<center><img src="./计算机网络第1章（概述）.assets/20201016103958.png" title="image-20201007005149993" width=30% /></center>
<ul>
<li>星型网络</li>
</ul>
<center><img src="./计算机网络第1章（概述）.assets/20201016104005.png" title="image-20201007005203129" width=25% /></center>
<ul>
<li>环形网络</li>
</ul>
<center><img src="./计算机网络第1章（概述）.assets/20201016104009.png" title="image-20201007005212345" width=30% /></center>
<ul>
<li>网状型网络</li>
</ul>
<center><img src="./计算机网络第1章（概述）.assets/20201016104015.png" title="image-20201007005249137" width=35% /></center>
<hr>
<h2 id="计算机网络的性能指标"> 计算机网络的性能指标</h2>
<h3 id="速率"> 速率</h3>
<center><img src="./计算机网络第1章（概述）.assets/20201016104021.png" width=80%></center>
<center><img src="./计算机网络第1章（概述）.assets/20201016104026.png" width=80%></center>
<h3 id="带宽"> 带宽</h3>
<center><img src="./计算机网络第1章（概述）.assets/20201016104031.png" width=80%></center>
<h3 id="吞吐量"> 吞吐量</h3>
<center><img src="./计算机网络第1章（概述）.assets/20201016104038.png" width=80%> </center>
<blockquote>
<p>带宽1 Gb/s的以太网，代表其额定速率是1 Gb/s，这个数值也是该以太网的<strong>吞吐量的绝对上限值</strong>。因此，对于带宽1 Gb/s的以太网，可能实际吞吐量只有 700 Mb/s，甚至更低。</p>
<p>注意：吞吐量还可以用每秒传送的字节数或帧数表示</p>
</blockquote>
<h3 id="时延"> 时延</h3>
<p>时延时指数据（一个报文或分组，甚至比特）从网络（或链路）的一端传送到另一端所需的时间。</p>
<p>网络时延由几部分组成：</p>
<ul>
<li>发送时延</li>
</ul>
<p>主机或路由器发送数据帧所需要的时间，也就是从发送数据帧的第一个比特算起，到该帧的最后一个比特发送完毕所需的时间。</p>
<ul>
<li>传播时延</li>
</ul>
<p>电磁波在信道中传播一定的距离需要花费的时间。</p>
<ul>
<li>处理时延</li>
</ul>
<p>主机或路由器在收到分组时要花费一定时间进行处理</p>
<ul>
<li>排队时延</li>
</ul>
<p>分组在进过网络传输时，要经过许多路由器。但分组在进入路由器后要先在输入队列中排队等待处理。</p>
<blockquote>
<p>有时会把排队时延看成<strong>处理时延 一部分</strong></p>
<p>总时延 = 发送时延 + 传播时延 + 处理时延 （处理时延 + 排队时延）</p>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016104045.png" width=80%> </center>
<center><img src="./计算机网络第1章（概述）.assets/20201016104050.png" width=80%> </center>
<blockquote>
<p>当处理时延忽略不计时，发送时延 和 传播时延谁占主导，要具体情况具体分析</p>
</blockquote>
<h3 id="时延带宽积"> 时延带宽积</h3>
<p>时延带宽积 = 传播时延 * 带宽</p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104057.png" width=80%> </center>
<h3 id="往返时间"> 往返时间</h3>
<p>互联网上的信息不仅仅单方向传输而是双向交互的。因此，我们有时很需要知道<strong>双向交互一次所需的时间</strong>。</p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104104.png" width=80%> </center>
<h3 id="利用率"> 利用率</h3>
<p>利用率有<strong>信道利用率</strong>和<strong>网络利用率</strong>两种。</p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104110.png" width=80%> </center>
<h3 id="丢包率"> 丢包率</h3>
<center><img src="./计算机网络第1章（概述）.assets/20201016104116.png" width=80%> </center>
<hr>
<h2 id="计算机网络体系结构"> 计算机网络体系结构</h2>
<h3 id="常见的计算机网络体系结构"> 常见的计算机网络体系结构</h3>
<center><img src="./计算机网络第1章（概述）.assets/20201016104122.png" width=80%></center>
<blockquote>
<p>如今用的最多的是TCP/IP体系结构，现今规模最大的、覆盖全球的、基于TCP/IP的互联网并未使用OSI标准。</p>
<p>TCP/IP体系结构相当于将OSI体系结构的<strong>物理层</strong>和<strong>数据链路层</strong>合并为了<strong>网络接口层</strong>，并去掉了<strong>会话层</strong>和<strong>表示层</strong>。</p>
<p>TCP/IP在网络层使用的协议是IP协议，IP协议的意思是网际协议，因此<strong>TCP/IP体系结构的网络层称为网际层</strong></p>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016104127.png" width=80%></center>
<blockquote>
<p>在用户主机的操作系统中，通常都带有符合TCP/IP体系结构标准的TCP/IP协议族。</p>
<p>而用于网络互连的路由器中，也带有符合TCP/IP体系结构标准的TCP/IP协议族。</p>
<p>只不过路由器一般只包含网络接口层和网际层。</p>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016104135.png" width=80%></center>
<blockquote>
<p><strong>网络接口层</strong>：并没有规定具体内容，这样做的目的是可以互连全世界各种不同的网络接口，例如：有线的以太网接口，无线局域网的WIFI接口等。</p>
<p><strong>网际层</strong>：它的核心协议是IP协议。</p>
<p><strong>运输层</strong>：TCP和UDP是这层的两个重要协议。</p>
<p><strong>应用层</strong>：这层包含了大量的应用层协议，如 HTTP , DNS 等。</p>
</blockquote>
<blockquote>
<p>**IP协议（网际层）<strong>可以将不同的</strong>网络接口（网络接口层）<strong>进行互连，并向其上的</strong>TCP协议和UDP协议（运输层）**提供网络互连服务</p>
<p>而<strong>TCP协议</strong>在享受IP协议提供的网络互连服务的基础上，可向<strong>应用层的相应协议</strong>提供<strong>可靠</strong>的传输服务。</p>
<p><strong>UDP协议</strong>在享受IP协议提供的网络互连服务的基础上，可向<strong>应用层的相应协议</strong>提供<strong>不可靠</strong>的传输服务。</p>
</blockquote>
<blockquote>
<p>TCP/IP体系结构中最重要的是<strong>IP协议</strong>和<strong>TCP协议</strong>，因此用TCP和IP来表示整个协议大家族。</p>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016104142.png" width=80%></center>
<blockquote>
<p>教学时把TCP/IP体系结构的<strong>网络接口层</strong>分成了<strong>物理层</strong>和<strong>数据链路层</strong></p>
</blockquote>
<h3 id="计算机网络体系结构分层的必要性"> 计算机网络体系结构分层的必要性</h3>
<center><img src="./计算机网络第1章（概述）.assets/20201016104151.png" width=80%></center
<p><strong>物理层问题</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104156.png" width=80%></center>
<blockquote>
<p>这图说明</p>
<ul>
<li>第一，严格来说，传输媒体并不属于物理层</li>
<li>计算机传输的信号，并不是图示的方波信号</li>
</ul>
<p>这样举例只是让初学者容易理解</p>
</blockquote>
<p><strong>数据链路层问题</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104204.png" width=80%> </center>
<p><strong>网络层问题</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104211.png" width=80%> </center>
<p><strong>运输层问题</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104216.png" width=80%> </center>
<blockquote>
<p>如何标识与网络通信相关的应用进程：一个分组到来，我们应该交给哪个进程处理呢？浏览器进程还是QQ进程</p>
</blockquote>
<p><strong>应用层问题</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104224.png" width=80%> </center>
<blockquote>
<p>应用层该用什么方法（应用层协议）去解析数据</p>
</blockquote>
<p><strong>总结</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104231.png" width=80%> </center>
<center><img src="./计算机网络第1章（概述）.assets/20201016104235.png" width=80%> </center>
<center><img src="./计算机网络第1章（概述）.assets/20201016104241.png" width=80%> </center>
<h3 id="计算机网络体系结构分层思想举例"> 计算机网络体系结构分层思想举例</h3>
<p>例子：主机的浏览器如何与Web服务器进行通信</p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104246.png" width=80%> </center>
<p><strong>解析：</strong></p>
<p>主机和Web服务器之间基于网络的通信，实际上是主机中的<strong>浏览器应用进程</strong>与Web服务器中的<strong>Web服务器应用进程</strong>之间基于<strong>网络的通信</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104254.png" width=80%> </center>
<p><strong>体系结构的各层在整个过程中起到怎样的作用？</strong></p>
<p><strong>1、发送方发送</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104300.png" width=80%> </center>
<blockquote>
<p>第一步：</p>
<ul>
<li><strong>应用层</strong>按照HTTP协议的规定构建一个<strong>HTTP请求报文</strong></li>
<li>应用层将<strong>HTTP请求报文</strong>交付给<strong>运输层</strong>处理</li>
</ul>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016104304.png" width=80%> </center>
<blockquote>
<p>第二步：</p>
<ul>
<li><strong>运输层</strong>给<strong>HTTP请求报文</strong>添加一个<strong>TCP首部</strong>，使之成为<strong>TCP报文段</strong></li>
<li><strong>TCP报文段的首部格式</strong>作用是区分应用进程以及实现可靠传输</li>
<li><strong>运输层</strong>将T<strong>CP报文段</strong>交付给<strong>网络层</strong>处理</li>
</ul>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016104309.png" width=80%> </center>
<blockquote>
<p>第三步：</p>
<ul>
<li><strong>网络层</strong>给<strong>TCP报文段</strong>添加一个<strong>IP首部</strong>，使之成为<strong>IP数据报</strong></li>
<li><strong>IP数据报的首部格式</strong>作用是使<strong>IP数据报</strong>可以在互联网传输，也就是被路由器转发</li>
<li><strong>网络层</strong>将<strong>IP数据报</strong>交付给<strong>数据链路层</strong>处理</li>
</ul>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016104314.png" width=80%> </center>
<blockquote>
<p>第四步：</p>
<ul>
<li><strong>数据链路层</strong>给<strong>IP数据报</strong>添加一个<strong>首部</strong>和一个<strong>尾部</strong>，使之成为<strong>帧</strong>     （图示右边为首部，左边为尾部）</li>
<li>该<strong>首部</strong>的作用主要是为了让<strong>帧</strong>能够在一段链路上或一个网络上传输，能够被相应的目的主机接收</li>
<li>该<strong>尾部</strong>的作用是让目的主机检查所接收到的<strong>帧</strong>是否有误码</li>
<li><strong>数据链路层</strong>将<strong>帧</strong>交付给<strong>物理层</strong></li>
</ul>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016104319.png" width=80%> </center>
<blockquote>
<p>第五步：</p>
<ul>
<li><strong>物理层</strong>先将<strong>帧</strong>看做是<strong>比特流</strong>，这里的网络N1假设是以太网，所以<strong>物理层</strong>还会给该<strong>比特流</strong>前面添加<strong>前导码</strong></li>
<li><strong>前导码</strong>的作用是为了让目的主机做好接收帧的准备</li>
<li><strong>物理层</strong>将装有<strong>前导码</strong>的<strong>比特流</strong>变换成相应的<strong>信号</strong>发送给传输媒体</li>
</ul>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016104325.png" width=80%> </center>
<blockquote>
<p>第六步：</p>
<ul>
<li><strong>信号</strong>通过<strong>传输媒体</strong>到达<strong>路由器</strong></li>
</ul>
</blockquote>
<p><strong>2、路由器转发</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104331.png" width=50% /></center>
<center><img src="计算机网络第1章（概述）.assets/20201016104337.png" title="image-20201007152138961" width=50% /></center>
<center><img src="计算机网络第1章（概述）.assets/20201016104350.png" title="image-20201007152627778" width=50% /></center>
<blockquote>
<p>在路由器中</p>
<ul>
<li><strong>物理层</strong>将<strong>信号</strong>变为<strong>比特流</strong>，然后去掉<strong>前导码</strong>后，将其交付给<strong>数据链路层</strong></li>
<li><strong>数据链路层</strong>将<strong>帧</strong>的<strong>首部</strong>和<strong>尾部</strong>去掉后，将其交付给<strong>网络层</strong>，这实际交付的是<strong>IP数据报</strong></li>
<li><strong>网络层</strong>解析<strong>IP数据报</strong>的<strong>首部</strong>，从中提取<strong>目的网络地址</strong></li>
</ul>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016104356.png" title="image-20201007152650863" width=50%/></center>
<center><img src="./计算机网络第1章（概述）.assets/20201016104400.png" title="image-20201007152812431" width=50% /></center>
<center><img src="./计算机网络第1章（概述）.assets/20201016104404.png" title="image-20201007153714840" width=50% /></center>
<blockquote>
<p>在路由器中</p>
<ul>
<li>提取<strong>目的网络地址</strong>后查找<strong>自身路由表</strong>。确定<strong>转发端口</strong>， 以便进行转发</li>
<li><strong>网络层</strong>将<strong>IP数据报</strong>交付给<strong>数据链路层</strong></li>
<li><strong>数据链路层</strong>给<strong>IP数据报</strong>添加一个<strong>首部</strong>和一个<strong>尾部</strong>，使之成为<strong>帧</strong></li>
<li><strong>数据链路层</strong>将帧交付给<strong>物理层</strong></li>
<li><strong>物理层</strong>先将<strong>帧</strong>看成<strong>比特流</strong>，这里的网络N2假设是以太网，所以<strong>物理层</strong>还会给该<strong>比特流</strong>前面添加<strong>前导码</strong></li>
<li>物理层将装有<strong>前导码</strong>的<strong>比特流</strong>变换成相应的<strong>信号</strong>发送给传输媒体，信号通过传输媒体到达<strong>Web服务器</strong></li>
</ul>
</blockquote>
<p><strong>3、接收方接收</strong></p>
<blockquote>
<p>和发送方（主机）发送过程的封装正好是反着来</p>
<p>在Web 服务器上</p>
<ul>
<li><strong>物理层</strong>将<strong>信号</strong>变换为<strong>比特流</strong>，然后去掉<strong>前导码</strong>后成为<strong>帧</strong>，交付给<strong>数据链路层</strong></li>
<li><strong>数据链路层</strong>将<strong>帧</strong>的<strong>首部</strong>和<strong>尾部</strong>去掉后成为<strong>IP数据报</strong>，将其交付给<strong>网络层</strong></li>
<li><strong>网络层</strong>将<strong>IP数据报</strong>的<strong>首部</strong>去掉后成为<strong>TCP报文段</strong>，将其交付给<strong>运输层</strong></li>
<li><strong>运输层</strong>将<strong>TCP报文段</strong>的<strong>首部</strong>去掉后成为<strong>HTTP请求报文</strong>，将其交付给<strong>应用层</strong></li>
<li><strong>应用层</strong>对<strong>HTTP请求报文</strong>进行<strong>解析</strong>，然后给主机发回<strong>响应报文</strong></li>
</ul>
<p><strong>发回响应报文的步骤和之前过程类似</strong></p>
</blockquote>
<center><img src="./计算机网络第1章（概述）.assets/20201016104410.png" width=80%> </center>
<h3 id="计算机网络体系结构中的专用术语"> 计算机网络体系结构中的专用术语</h3>
<p>以下介绍的专用术语来源于OSI的七层协议体系结构，但也适用于TCP/IP的四层体系结构和五层协议体系结构</p>
<p><strong>实体</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104417.png" width=80%> </center>
<p><strong>协议</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104423.png" width=80%> </center>
<blockquote>
<p>协议：控制两个对等实体进行逻辑通信的规则的集合</p>
<p>协议三要素：</p>
<ul>
<li>语法：定义所交换信息的格式</li>
<li>语义：定义收发双方所要完成的操作</li>
<li>同步：定义收发双发的时序关系</li>
</ul>
</blockquote>
<p><strong>服务</strong></p>
<center><img src="./计算机网络第1章（概述）.assets/20201016104434.png" width=80%> </center>
<center><img src="./计算机网络第1章（概述）.assets/20201016104757.png" width=80%> </center>
<center><img src="./计算机网络第1章（概述）.assets/20201016104806.png" width=80%> </center>
<blockquote>
<p>本文摘自：https://github.com/BloothOfYouth/Computer-Network-Notes</p>
<p>参考书籍：谢希仁《计算机网络》</p>
<p>该笔记参考：湖科大教书匠《计算机网络》微课堂：https://www.bilibili.com/video/BV1c4411d7jb</p>
</blockquote>
]]></content:encoded>
    </item>
    <item>
      <title>docker使用</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/docker%E4%BD%BF%E7%94%A8/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/docker%E4%BD%BF%E7%94%A8/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">docker使用</source>
      <category>MachineLearning</category>
      <pubDate>Thu, 24 Mar 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h1 id="docker简介"> Docker简介</h1>
<p>Docker使用Google公司推出的Go语言开发实现，基于Linux内核的cgroup，namespace，以及OverlayFS类的Union FS等技术，对进程进行封装隔离，属于<strong>操作系统层面的虚拟化技术</strong>。隔离的进程独立于宿主和其它的隔离进程，因此也称为<strong>容器</strong>。</p>
<p>Docker在容器的基础上进一步进行封装，从文件系统、网络互联到进程隔离等，极大简化了容器的创建和维护，使得Docker比虚拟机技术更为轻便、快捷。</p>
<h1 id="docker优势"> Docker优势</h1>
<p>（1）<strong>系统资源更高效的利用</strong></p>
<p>容易不需要进行硬件虚拟以及运行完整操作系统等额外开销，Docker对系统资源的利用率更高，一个相同配置的主机，往往可以运行更多数量的应用。</p>
<p>（2）<strong>启动时间更快</strong></p>
<p>Docker容器直接运行于宿主内核，无需启动完整的操作系统，可以做到秒、毫秒级的启动时间，大大缩短开发、测试和部署的时间。</p>
<p>（3）<strong>运行环境的一致性</strong></p>
<p>Docker提供除内核外完整的运行环境，确保应用运行环境的一致性。</p>
<p>（4）<strong>持续交付和部署</strong></p>
<p>对开发和运维（DevOps）人员来说，最希望的是一次创建或配置就能实现在任何地方正常运行。Docker通过定制<strong>应用镜像</strong>来实现持续集成、交付和部署。开发人员通过<strong>Dockerfile</strong>构建镜像，结合<strong>持续集成</strong>（Continuous Integration）系统进行集成测试；运维人员直接在生产环境中快速部署该镜像，或者结合持续部署（Continuous Delivery/Deployment）系统进行自动部署。使用Dockerfile使得镜像构建透明化。</p>
<p>（5）<strong>迁移轻松</strong></p>
<p>Docker的使用确保了执行环境的一致性，便于多平台应用迁移。</p>
<p>（6）<strong>便于维护和扩展</strong></p>
<p>Docker使用分层存储及镜像技术，使得应用重复部分的复用更容易。Docker团队同各个开源项目团队一起维护了一大批高质量的官方镜像。</p>
<h1 id="基本概念"> 基本概念</h1>
<h2 id="镜像-image"> 镜像（Image）</h2>
<p>Docker的镜像相当于一个<strong>root文件系统</strong>（Linux内核启动后，会挂载root文件系统为其提供用户空间支持）。</p>
<p>Docker镜像除提供容器运行时<strong>所需的程序、库、资源、配置等文件</strong>外，还包括运行时的一些<strong>配置</strong>参数。镜像不包含任何动态数据，其内容在构建之后不会改变。</p>
<h2 id="分层存储"> 分层存储</h2>
<p>Docker镜像包含操作系统完整的root文件系统，其体积往往是庞大的，在Docker设计中充分利用**Union FS（联合文件系统）**技术，设计成分层存储架构，使得镜像并非是一个ISO那样的打包文件，实际是一组文件系统的组成（多层文件系统的联合）。</p>
<blockquote>
<p>Union FS：2004年纽约州立石溪分校开发，可以把多个目录（分支）内容联合挂载到同一个目录下，目录的物理位置是分开的。Union FS运行只读和可读写目录，可以同时删除和增加内容。Union FS的一个典型应用可以把多张CD光盘合并成一个统一的光盘目录（归档）。</p>
</blockquote>
<h2 id="容器-container"> 容器（Container）</h2>
<p>镜像和容器的关系，就像是面向对象程序设计中的<strong>类</strong>和<strong>实例</strong>，镜像是静态的定义，容器是镜像运行时的实体。</p>
<p>容器可以被创建、启动、停止、删除、暂停等。其实质是进程，与宿主执行的进程不同的是，容器进程运行属于自己的独立命名空间，因此容器可以拥有自己的root文件系统、网络配置、进程空间、用户ID空间等。</p>
<p>容器内的进程是运行在一个隔离的环境，独立于宿主系统。这使得容器封装的应用比直接在宿主运行更安全。</p>
<blockquote>
<p>容器和虚拟机是非常相似的资源虚拟化技术。</p>
<p>虚拟化是一个过程，将RAM、CPU、磁盘或网络等系统单一资源虚拟化成多个资源。</p>
<p>虚拟机将整个计算机虚拟化到硬件层，容器只虚拟化操作系统级别以上的软件层。</p>
</blockquote>
<h2 id="仓库-repository"> 仓库（Repository）</h2>
<p>镜像构建完成后，可以很容易在当前宿主机上运行，但如果需要在其它服务器上使用该镜像，需要一个集中的存储、分发镜像的服务，即<strong>Docker Registry</strong>。</p>
<p>一个Docker Registry可以包含多个<strong>仓库</strong>；每一个仓库可以包含多个<strong>标签（tag）</strong>；每一个标签对应一个镜像。</p>
<p>通常，一个仓库包含同一个软件不同版本的镜像，标签常用于对应该软件的各个版本。通过<code>&lt;仓库名&gt;:&lt;标签&gt;</code>格式指定具体是这个软件哪个版本的镜像。如果不指定，则以<code>latest</code>作为默认标签。</p>
<blockquote>
<p>以Ubuntu镜像为例，ubuntu是仓库名，包含不同版本的标签，如16.04，18.04等。</p>
</blockquote>
<p>仓库名经常以<strong>两段式</strong>路径出现，如<code>jwilder/nginx-proxy</code>，前者往往意味着Docker Registry多用户环境下的用户名，后者是对应的软件名，并非绝对。</p>
<h2 id="docker-registry服务"> Docker Registry服务</h2>
<p><strong>公开服务</strong></p>
<p>开发给用户使用，允许用户管理镜像的Registry服务。常用的Registry公开服务包括官方的Docker Hub（默认的Registry），Red Hat的Quary.io，GitHub推出的ghcr.io等。</p>
<p>国内访问通过阿里云加速器等，可以更快的直接从网内地址下载Docker Hub的镜像。</p>
<p><strong>私有服务</strong></p>
<p>用户可以在本地搭建私有的Docker Registry。</p>
<p><strong>第三方软件</strong></p>
<p>开源的Docker Registry镜像只提供Docker Registry API的服务端实现，足以支持docker命令，但不含图像界面，镜像维护，用户管理，访问控制等功能。</p>
<p>第三方软件实现的Docker Registry API提供包括用户界面以及一些高级功能。如Harbor和Sonatype Nexus。</p>
]]></content:encoded>
    </item>
    <item>
      <title>最小二乘法(LSM)</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/mathematics/%E6%9C%80%E5%B0%8F%E4%BA%8C%E4%B9%98%E6%B3%95/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/mathematics/%E6%9C%80%E5%B0%8F%E4%BA%8C%E4%B9%98%E6%B3%95/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">最小二乘法(LSM)</source>
      <description>关于最小二乘法（least-squares method）。</description>
      <category>MachineLearning</category>
      <pubDate>Tue, 01 Mar 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="原理"> 原理</h2>
<p>最小二乘法（least-squares method）是由勒让德在19世纪发现的，形式如下：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>目标函数（误差函数）</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.6em;vertical-align:-0.55em;"></span><span style="position:relative;top:0em;">∑</span><span>(</span><span>观测值</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.1141em;vertical-align:-0.25em;"></span><span>理论值</span><span><span>)</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></span></p>
<p>其中，观测值是多组样本，理论值是假设的拟合函数。使用最小二乘法的目标是使得误差函数最小化时的拟合函数模型。</p>
<p>假设给定一个训练集<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4444em;"></span><span>x</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.0913em;vertical-align:-0.25em;"></span><span>(</span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.10903em;">N</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>)</span><span><span><span><span style="height:0.8413em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span></span></span></span>，由<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span>的<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">N</span></span></span></span>次观测组成，对应的观测值为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6349em;"></span><span>t</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.0913em;vertical-align:-0.25em;"></span><span>(</span><span><span>t</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>t</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>t</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.10903em;">N</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>)</span><span><span><span><span style="height:0.8413em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span></span></span></span>。给出一个多项式拟合函数（polynomial fitting function）：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.03588em;">y</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.01597em;">w</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7333em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>0</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.7333em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>x</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>⋯</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.0413em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.10903em;">M</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>x</span><span><span><span><span style="height:0.8913em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.10903em;">M</span></span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:3.2421em;vertical-align:-1.4138em;"></span><span><span><span><span style="height:1.8283em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span style="margin-right:0.05724em;">j</span><span>=</span><span>0</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span style="margin-right:0.10903em;">M</span></span></span></span></span><span>​</span></span><span><span style="height:1.4138em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span><span>x</span><span><span><span><span style="height:0.8747em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span></span></span></span></span></span></span></span></span></p>
<p>其中，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">M</span></span></span></span>是多项式的阶数（order），多项式系数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.02691em;">w</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0269em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.10903em;">M</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>整体用向量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4444em;"></span><span style="margin-right:0.01597em;">w</span></span></span></span>表示，最小二乘法要找到一组<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4444em;"></span><span style="margin-right:0.01597em;">w</span></span></span></span>，使得误差函数（error function）：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>min</span></span></span><span style="margin-right:0.05764em;">E</span><span>(</span><span style="margin-right:0.01597em;">w</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:3.0954em;vertical-align:-1.2671em;"></span><span><span><span>min</span></span></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>2</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.1667em;"></span><span><span><span><span style="height:1.8283em;"><span style="top:-1.8829em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>n</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span style="margin-right:0.10903em;">N</span></span></span></span><span>​</span></span><span><span style="height:1.2671em;"><span></span></span></span></span></span><span>{</span><span style="margin-right:0.03588em;">y</span><span>(</span><span><span>x</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.01597em;">w</span><span>)</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.1141em;vertical-align:-0.25em;"></span><span><span>t</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>}</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span>（残差平方和最小）</span></span></span></span></span></p>
<p>其中，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7651em;vertical-align:-0.15em;"></span><span><span>t</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>是对应的目标值，系数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1901em;vertical-align:-0.345em;"></span><span><span></span><span><span><span><span style="height:0.8451em;"><span style="top:-2.655em;"><span style="height:3em;"></span><span><span><span>2</span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.394em;"><span style="height:3em;"></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.345em;"><span></span></span></span></span></span><span></span></span></span></span></span>引入是为了求导后系数变为1，便于计算。</p>
<p>误差函数是关于<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4444em;"></span><span style="margin-right:0.01597em;">w</span></span></span></span>的二次函数（quadratic function），其导数是关于<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4444em;"></span><span style="margin-right:0.01597em;">w</span></span></span></span>的线性函数，所以误差函数的最小值有唯一解<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6887em;"></span><span><span style="margin-right:0.01597em;">w</span><span><span><span><span style="height:0.6887em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>∗</span></span></span></span></span></span></span></span></span></span></span>。</p>
<blockquote>
<p>类似多项式函数关于未知参数满足线性关系的函数具有重要性质，该模型称为<strong>线性模型</strong>。</p>
<p>最小平方问题分为两种：线性最小二乘和非线性最小二乘法。取决于在所有未知数的残差是否为线性。线性的最小平方问题用在回归分析中，有一个封闭形式的解决方案；非线性的问题通常用迭代细化解决，在每一次迭代中，系统由线性近似。两者核心演算相同。</p>
</blockquote>
<p>该误差函数最小值为非负量，当且仅当函数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.03588em;">y</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.01597em;">w</span><span>)</span></span></span></span>对所有训练数据点均做出正确预测时，误差函数为0。</p>
<center><img src="./LSM/error function.png"  width=50% ></center>
<h2 id="线性最小二乘法矩阵求解"> 线性最小二乘法矩阵求解</h2>
<p>假设样本模型：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">XB</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.22222em;">Y</span></span></span></span></span></p>
<p>定义误差函数：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.09618em;">J</span><span>(</span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.0074em;vertical-align:-0.686em;"></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>2</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span>(</span><span style="margin-right:0.05017em;">XB</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.1413em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">Y</span><span><span>)</span><span><span><span><span style="height:0.8913em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span><span>(</span><span style="margin-right:0.05017em;">XB</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">Y</span><span>)</span></span></span></span></span></p>
<p>根据最小二乘法原理，对该损失函数对<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">B</span></span></span></span>求偏导（partial derivation）并取0，有：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">B</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.1413em;vertical-align:-0.25em;"></span><span>(</span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.8913em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span><span><span>)</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>−</span><span>1</span></span></span></span></span></span></span></span></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.8913em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span><span style="margin-right:0.22222em;">Y</span></span></span></span></span></p>
<p>矩阵乘法：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">XB</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7073em;vertical-align:-0.024em;"></span><span style="margin-right:0.22222em;">Y</span><span style="margin-right:0.2778em;"></span><span>⟹</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8913em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.8913em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span><span style="margin-right:0.05017em;">XB</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.9153em;vertical-align:-0.024em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.8913em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span><span style="margin-right:0.22222em;">Y</span><span style="margin-right:0.2778em;"></span><span>⟹</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">B</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.1413em;vertical-align:-0.25em;"></span><span>(</span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.8913em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span><span><span>)</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>−</span><span>1</span></span></span></span></span></span></span></span></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.8913em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span><span style="margin-right:0.22222em;">Y</span></span></span></span></span></p>
<h2 id="损失函数和误差函数区别"> 损失函数和误差函数区别</h2>
<p>loss function和error function区别：</p>
<blockquote>
<p>Error is the difference between the actual/true value <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span style="margin-right:0.02778em;">θ</span></span></span></span> and the predicted/estimated value <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.9579em;"></span><span><span><span><span style="height:0.9579em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.02778em;">θ</span></span><span style="top:-3.2634em;"><span style="height:3em;"></span><span style="left:-0.1667em;"><span>^</span></span></span></span></span></span></span></span></span></span>.</p>
<p>Loss is a measurement of how well a model fits the 'data'.</p>
</blockquote>
<p>error是表示<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span style="margin-right:0.02778em;">θ</span></span></span></span>和<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.9579em;"></span><span><span><span><span style="height:0.9579em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.02778em;">θ</span></span><span style="top:-3.2634em;"><span style="height:3em;"></span><span style="left:-0.1667em;"><span>^</span></span></span></span></span></span></span></span></span></span>的差值，定义不变；loss值和选择的loss function有关。如何选择loss function取决于数据和目标。</p>
<p>常用的损失函数有：</p>
<p>1、Mean Absolute Error(MAE) - average of the errors</p>
<blockquote>
<p>也称为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>L</span><span>1</span></span></span></span> Loss</p>
</blockquote>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>L</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.9535em;vertical-align:-1.3021em;"></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>n</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.1667em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8479em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span style="margin-right:0.03148em;">k</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.3021em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span>∣</span><span style="margin-right:0.02778em;">θ</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.2079em;vertical-align:-0.25em;"></span><span><span><span><span style="height:0.9579em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.02778em;">θ</span></span><span style="top:-3.2634em;"><span style="height:3em;"></span><span style="left:-0.1667em;"><span>^</span></span></span></span></span></span></span><span>∣</span></span></span></span></span></p>
<p>2、Mean Squared Error(MSE)</p>
<blockquote>
<p>也称为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>L</span><span>2</span></span></span></span> Loss</p>
</blockquote>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>L</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.9535em;vertical-align:-1.3021em;"></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>n</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.1667em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8479em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span style="margin-right:0.03148em;">k</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.3021em;"><span></span></span></span></span></span><span>(</span><span style="margin-right:0.02778em;">θ</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.2079em;vertical-align:-0.25em;"></span><span><span><span><span style="height:0.9579em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.02778em;">θ</span></span><span style="top:-3.2634em;"><span style="height:3em;"></span><span style="left:-0.1667em;"><span>^</span></span></span></span></span></span></span><span><span>)</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></span></p>
<p>3、<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.9694em;vertical-align:-0.2861em;"></span><span><span>L</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>p</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span></span></span></span> - MAE and MSE apply power(幂) of 1 and 2 to the errors, respectively, and average them. Because of this, they also called <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>L</span><span>1</span></span></span></span> and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>L</span><span>2</span></span></span></span> loss functions. However, you can apply higher(or negative) powers, or <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>p</span></span></span></span>, to the errors.</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.9694em;vertical-align:-0.2861em;"></span><span><span>L</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>p</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.9535em;vertical-align:-1.3021em;"></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>n</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.1667em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8479em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span style="margin-right:0.03148em;">k</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.3021em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span>∣</span><span style="margin-right:0.02778em;">θ</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.2079em;vertical-align:-0.25em;"></span><span><span><span><span style="height:0.9579em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="margin-right:0.02778em;">θ</span></span><span style="top:-3.2634em;"><span style="height:3em;"></span><span style="left:-0.1667em;"><span>^</span></span></span></span></span></span></span><span><span>∣</span><span><span><span><span style="height:0.7144em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>p</span></span></span></span></span></span></span></span></span></span></span></span></p>
<p>error总体上涉及真实值对于预测值或者期望值的偏离，loss是一个关于模型产生误差的程度的量化方法，是由不准确的预测所导致的负面结果。</p>
<blockquote>
<p>误差函数error是一个统计学对象，用于<strong>参数估计</strong>，属于估计理论范畴。 误差函数的最小化常也称为最小二乘法，是一种参数估计方法。</p>
<p>最小二乘法与高斯模型下的极大似然估计的结果一致。</p>
<p>损失函数loss用于<strong>预测</strong>，属于决策理论范畴。</p>
<p>来源：https://gaolei786.github.io/statistics/error.html</p>
</blockquote>
<h2 id="参考"> 参考</h2>
<p>https://zhuanlan.zhihu.com/p/38128785</p>
<p>《Pattern Recognition and Machine Learning-PRML》</p>
]]></content:encoded>
    </item>
    <item>
      <title>线性代数-随机变量的数字特征</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/mathematics/%E7%BA%BF%E6%80%A7%E4%BB%A3%E6%95%B0_1/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/mathematics/%E7%BA%BF%E6%80%A7%E4%BB%A3%E6%95%B0_1/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">线性代数-随机变量的数字特征</source>
      <description>随机变量的数字特征，包括期望、方差、协方差、协方差矩阵、中心矩等的简介。</description>
      <category>Maths</category>
      <pubDate>Thu, 03 Mar 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="基向量"> 基向量</h2>
<p>设<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">B</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span>e</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>e</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>e</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>是在数域<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">F</span></span></span></span>上的向量空间<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.22222em;">V</span></span></span></span>的有限子集，若满足下列条件：</p>
<p>（1）对任意<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>λ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span><span style="margin-right:0.13889em;">F</span><span><span><span><span style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span></span></span></span></span></span></span></span>，如果<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8444em;vertical-align:-0.15em;"></span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>e</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8444em;vertical-align:-0.15em;"></span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>e</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>⋯</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8444em;vertical-align:-0.15em;"></span><span><span>λ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>e</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span>，则必然<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8444em;vertical-align:-0.15em;"></span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8444em;vertical-align:-0.15em;"></span><span><span>λ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.3669em;"></span><span>⋯</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8444em;vertical-align:-0.15em;"></span><span><span>λ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span>。</p>
<p>即：<strong>基向量必定是一组线性无关向量组</strong>。</p>
<p>（2）对任意<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span style="margin-right:0.03588em;">v</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.22222em;">V</span></span></span></span>，可以选择<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span><span>μ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>μ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>μ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span><span style="margin-right:0.13889em;">F</span><span><span><span><span style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span></span></span></span></span></span></span></span>，使得<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.03588em;">v</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7778em;vertical-align:-0.1944em;"></span><span><span>μ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>e</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.7778em;vertical-align:-0.1944em;"></span><span><span>μ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>e</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>⋯</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span>μ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>e</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>。</p>
<p>即：<strong>向量空间<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.22222em;">V</span></span></span></span>内的任意向量都可以由其中的任意一组基向量（基向量不唯一）线性表示</strong>。</p>
<p>则<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">B</span></span></span></span>是向量空间<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.22222em;">V</span></span></span></span>的一组基。</p>
<h2 id="标准正交基"> 标准正交基</h2>
<p><strong>几何定义</strong>：</p>
<p>存在向量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.0037em;">α</span></span></span></span>和<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span style="margin-right:0.05278em;">β</span></span></span></span>如果垂直，即正交，满足：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4445em;"></span><span style="margin-right:0.0037em;">α</span><span style="margin-right:0.2222em;"></span><span>⋅</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span style="margin-right:0.05278em;">β</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>∣</span><span style="margin-right:0.0037em;">α</span><span>∣∣</span><span style="margin-right:0.05278em;">β</span><span>∣</span><span style="margin-right:0.1667em;"></span><span>cos</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">θ</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span></span></p>
<p><strong>代数定义</strong>：</p>
<p>两个向量的内积（点积、标量积）为零，即：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4445em;"></span><span style="margin-right:0.0037em;">α</span><span style="margin-right:0.2222em;"></span><span>⋅</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span style="margin-right:0.05278em;">β</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span style="margin-right:0.0037em;">α</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0037em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span style="margin-right:0.05278em;">β</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0528em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>⋯</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span style="margin-right:0.0037em;">α</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0037em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span style="margin-right:0.05278em;">β</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0528em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span></span></p>
<p><strong>标准正交基</strong>：</p>
<p>若一组正交基内的基向量的模长都是单位长度1，称为标准正交基或规范正交基。</p>
<h2 id="随机变量"> 随机变量</h2>
<p>一个从<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>Ω</span></span></span></span>映射到另一个集合（一般是实数域<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.00773em;">R</span></span></span></span>）的函数，必须是一个<strong>可测函数</strong>，若<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>是一个实数随机变量，使得<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>为正的样本输出的集合<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span style="margin-right:0.03588em;">ω</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span>Ω</span><span style="margin-right:0.2778em;"></span><span>:</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.07847em;">X</span><span>(</span><span style="margin-right:0.03588em;">ω</span><span>)</span><span style="margin-right:0.2778em;"></span><span>&gt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>0</span><span>}</span></span></span></span>是一个<strong>事件</strong>，通常只写作<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>&gt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>0</span><span>}</span></span></span></span>，概率<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>({</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>&gt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>0</span><span>})</span></span></span></span>写为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>&gt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>0</span><span>)</span></span></span></span>。</p>
<p><font size =5><strong>独立</strong></font></p>
<p>若<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span>A</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span>A</span><span>)</span><span style="margin-right:0.13889em;">P</span><span>(</span><span style="margin-right:0.05017em;">B</span><span>)</span></span></span></span>，则两个事件是独立的。</p>
<p>若任何与随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>有关的事件和任何与随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.22222em;">Y</span></span></span></span>有关的事件独立，则<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span></span></span></span>两个随机变量是独立的。</p>
<p><font size =5><strong>互斥</strong></font></p>
<p>若<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span>A</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span>，则<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span>A</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.05017em;">B</span></span></span></span>两个事件互斥或不相交。</p>
<p>若两个事件<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span>A</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.05017em;">B</span></span></span></span>互斥，则<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span>A</span><span style="margin-right:0.2222em;"></span><span>∪</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span>A</span><span>)</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span style="margin-right:0.05017em;">B</span><span>)</span></span></span></span>，可以扩展为有限个或无数个事件组成的序列。</p>
<blockquote>
<p><strong>概率空间</strong>：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span>Ω</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.13889em;">F</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.13889em;">P</span><span>)</span></span></span></span>是一个总测度为1的测度空间（<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span>Ω</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>1</span><span>)</span></span></span></span>），<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span>Ω</span><span>)</span></span></span></span>是一个非空集合，称为“样本空间”，集合中元素称为“样本输出”；<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">F</span></span></span></span>是样本空间<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>Ω</span></span></span></span>的一个非空子集，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">F</span></span></span></span>中元素称为事件。</p>
<p><strong>可测函</strong>数：是可测空间内的保持结构的函数。</p>
<p><strong>可测空间</strong>：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span>Ω</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.13889em;">F</span><span>)</span></span></span></span>，</p>
<p><strong>事件</strong>：是样本输出的集合，在此集合上可定义其概率</p>
<p><strong><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">P</span></span></span></span></strong>：概率，也称概率测度。是从集合<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">F</span></span></span></span>到实数域<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.00773em;">R</span></span></span></span>的函数，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">P</span><span style="margin-right:0.2778em;"></span><span>:</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">F</span><span style="margin-right:0.2778em;"></span><span>→</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.00773em;">R</span></span></span></span>，每个事件都被此函数赋予一个0~1之间的概率值</p>
<p><strong>概率质量函数（probability mass function, pmf）</strong>：离散随机变量在各个特定取值上的概率。</p>
<p><strong>概率密度函数</strong>：本身不是概率，只有对连续随机变量的概率密度函数在某区间内进行积分才是概率。</p>
</blockquote>
<h2 id="期望-expectation"> 期望（expectation）</h2>
<p>期望是一个离散型随机变量输出值的加权平均，是试验中每次可能结果乘以其结果概率的总和。</p>
<p>在统计学中，估算变量的期望值时，经常用到的方法是重复测量此变量的值，再用所得的数据的均值估计此变量的期望值。</p>
<p>在概率分布中，期望和方差/标准差是一种分布的重要特征。</p>
<p><font size =5><strong>定义</strong></font></p>
<p>如果<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>是在<strong>概率空间</strong><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span>Ω</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.13889em;">F</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.13889em;">P</span><span>)</span></span></span></span>中的<strong>随机变量</strong>，期望值<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span>]</span></span></span></span>定义为：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span>]</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.2719em;vertical-align:-0.9119em;"></span><span><span style="margin-right:0.44445em;position:relative;top:-0.0011em;">∫</span><span><span><span><span style="height:-0.4336em;"><span style="top:-1.7881em;margin-left:-0.4445em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>Ω</span></span></span></span><span>​</span></span><span><span style="height:0.9119em;"><span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.07847em;">X</span><span> </span><span><span><span>d</span></span></span><span style="margin-right:0.13889em;">P</span></span></span></span></span></p>
<p>如果<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>是<strong>离散</strong>的随机变量，输出值为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span></span></span></span>和输出值相应的概率为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span>p</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>p</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span></span></span></span>（<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.7277em;vertical-align:-0.9777em;"></span><span><span><span><span style="height:0.75em;"><span style="top:-2.1223em;margin-left:0em;"><span style="height:3em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>∑</span></span></span></span><span>​</span></span><span><span style="height:0.9777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span>p</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>1</span></span></span></span>），级数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.7277em;vertical-align:-0.9777em;"></span><span><span><span><span style="height:0.75em;"><span style="top:-2.1223em;margin-left:0em;"><span style="height:3em;"></span><span><span><span>i</span></span></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>∑</span></span></span></span><span>​</span></span><span><span style="height:0.9777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span>p</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span><strong>绝对收敛</strong>，期望值为：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span>]</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.3277em;vertical-align:-1.2777em;"></span><span><span><span><span style="height:1.05em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span>p</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span></span></p>
<p>如果<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>是<strong>连续</strong>的随机变量，存在一个相应的<strong>概率密度函数</strong><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10764em;">f</span><span>(</span><span>x</span><span>)</span></span></span></span>，积分<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.2734em;vertical-align:-0.4142em;"></span><span><span style="margin-right:0.19445em;position:relative;top:-0.0006em;">∫</span><span><span><span><span style="height:0.8593em;"><span style="top:-2.3442em;margin-left:-0.1945em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>−</span><span>∞</span></span></span></span><span style="top:-3.2579em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>∞</span></span></span></span></span><span>​</span></span><span><span style="height:0.4142em;"><span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span>x</span><span style="margin-right:0.10764em;">f</span><span>(</span><span>x</span><span>)</span><span>d</span><span>x</span></span></span></span>是<strong>绝对收敛</strong>，期望值为：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span>]</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.3846em;vertical-align:-0.9703em;"></span><span><span style="margin-right:0.44445em;position:relative;top:-0.0011em;">∫</span><span><span><span><span style="height:1.4143em;"><span style="top:-1.7881em;margin-left:-0.4445em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>−</span><span>∞</span></span></span></span><span style="top:-3.8129em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>∞</span></span></span></span></span><span>​</span></span><span><span style="height:0.9703em;"><span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span>x</span><span style="margin-right:0.10764em;">f</span><span>(</span><span>x</span><span>)</span><span>d</span><span>x</span></span></span></span></span></p>
<p><font size =5><strong>性质</strong></font></p>
<p>1、线性函数</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[</span><span>a</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">bY</span><span>]</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>a</span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span>]</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>b</span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.22222em;">Y</span><span>]</span></span></span></span></span></p>
<p>其中：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span></span></span></span>是同一概率空间的两个随机变量（独立或不独立），<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span>a</span><span>,</span><span style="margin-right:0.1667em;"></span><span>b</span></span></span></span>为任意实数。</p>
<p>2、一个随机变量的函数的期望值不等于这个随机变量的期望值的函数</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.03588em;">g</span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)]</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.2719em;vertical-align:-0.9119em;"></span><span><span style="margin-right:0.44445em;position:relative;top:-0.0011em;">∫</span><span><span><span><span style="height:-0.4336em;"><span style="top:-1.7881em;margin-left:-0.4445em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>Ω</span></span></span></span></span><span>​</span></span><span><span style="height:0.9119em;"><span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">g</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.10764em;">f</span><span>(</span><span>x</span><span>)</span><span>d</span><span>x</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span><span></span></span></span><span></span></span></span></span></span><span>=</span></span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.03588em;">g</span><span>(</span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span>])</span></span></span></span></span></p>
<p>3、一般，两个随机变量的积的期望值不等于这两个随机变量的期望值的积。</p>
<p>当<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.22222em;">Y</span><span>]</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span>]</span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.22222em;">Y</span><span>]</span></span></span></span>成立时，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span></span></span></span>独立，<strong>协方差</strong>为0。当两个随机变量独立时，它们的协方差（若存在）为0。</p>
<h2 id="方差-variance"> 方差（variance）</h2>
<p>概率论和统计学中，方差用于描述一个随机变量的离散程度，即该变量距离期望值的距离。</p>
<p>一个实随机变量的方差也成为<strong>二阶中心矩</strong>，即各个误差平方和再除以总数。</p>
<p>方差的正平方根称为标准差。</p>
<p><font size =5><strong>定义</strong></font></p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>是服从分布<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">F</span></span></span></span>的随机变量，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span>]</span></span></span></span>是期望，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>μ</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span>]</span></span></span></span>，随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>或者分布<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">F</span></span></span></span>的方差为：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[(</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.1141em;vertical-align:-0.25em;"></span><span>μ</span><span><span>)</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span>]</span></span></span></span></span></p>
<p>方差与协方差关系：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.07847em;">X</span><span>)</span></span></span></span></span></p>
<p>方差可以表示为“平方的期望减去期望的平方”：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.1141em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[</span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>2</span><span style="margin-right:0.07847em;">X</span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span>]</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.1141em;vertical-align:-0.25em;"></span><span>(</span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span>]</span><span><span>)</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span>]</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.1141em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[</span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span>]</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.1141em;vertical-align:-0.25em;"></span><span>(</span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span>]</span><span><span>)</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span></span></p>
<p>离散随机变量的方差：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:6.4581em;vertical-align:-2.9791em;"></span><span><span><span><span><span style="height:3.4791em;"><span style="top:-5.4791em;"><span style="height:3.6514em;"></span><span></span></span><span style="top:-2.25em;"><span style="height:3.6514em;"></span><span></span></span></span><span>​</span></span><span><span style="height:2.9791em;"><span></span></span></span></span></span><span><span><span><span style="height:3.4791em;"><span style="top:-5.4791em;"><span style="height:3.6514em;"></span><span><span></span><span><span><span>Var</span><span>(</span><span>X</span><span>)</span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span>p</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>⋅</span><span style="margin-right:0.2222em;"></span><span>(</span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span>μ</span><span><span>)</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span>(</span><span><span>p</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>⋅</span><span style="margin-right:0.2222em;"></span><span><span>x</span><span><span><span><span style="height:0.8641em;"><span style="top:-2.453em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.247em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span><span>μ</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span style="top:-2.25em;"><span style="height:3.6514em;"></span><span><span></span><span>s</span><span>.</span><span>t</span><span>.</span><span> </span><span>μ</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span>p</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>⋅</span><span style="margin-right:0.2222em;"></span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:2.9791em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:3.4791em;"><span style="top:-5.4791em;"><span style="height:3.6514em;"></span><span></span></span><span style="top:-2.25em;"><span style="height:3.6514em;"></span><span></span></span></span><span>​</span></span><span><span style="height:2.9791em;"><span></span></span></span></span></span></span></span></span></p>
<p>连续随机变量的方差：</p>
<p>随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>是连续分布且存在期望，对应的概率密度函数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10764em;">f</span><span>(</span><span>x</span><span>)</span></span></span></span>，</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:5.0445em;vertical-align:-2.2722em;"></span><span><span><span><span><span style="height:2.7723em;"><span style="top:-4.7723em;"><span style="height:3.36em;"></span><span></span></span><span style="top:-2.25em;"><span style="height:3.36em;"></span><span></span></span></span><span>​</span></span><span><span style="height:2.2722em;"><span></span></span></span></span></span><span><span><span><span style="height:2.7723em;"><span style="top:-4.7723em;"><span style="height:3.36em;"></span><span><span></span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span style="margin-right:0.44445em;position:relative;top:-0.0011em;">∫</span><span>(</span><span>x</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span>μ</span><span><span>)</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span style="margin-right:0.10764em;">f</span><span>(</span><span>x</span><span>)</span><span>d</span><span>x</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span style="margin-right:0.44445em;position:relative;top:-0.0011em;">∫</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span style="margin-right:0.10764em;">f</span><span>(</span><span>x</span><span>)</span><span>d</span><span>x</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span><span>μ</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span style="top:-2.25em;"><span style="height:3.36em;"></span><span><span></span><span>s</span><span>.</span><span>t</span><span>.</span><span> </span><span>μ</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span style="margin-right:0.44445em;position:relative;top:-0.0011em;">∫</span><span style="margin-right:0.1667em;"></span><span>x</span><span style="margin-right:0.10764em;">f</span><span>(</span><span>x</span><span>)</span><span>d</span><span>x</span></span></span></span><span>​</span></span><span><span style="height:2.2722em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:2.7723em;"><span style="top:-4.7723em;"><span style="height:3.36em;"></span><span></span></span><span style="top:-2.25em;"><span style="height:3.36em;"></span><span></span></span></span><span>​</span></span><span><span style="height:2.2722em;"><span></span></span></span></span></span></span></span></span></p>
<p><font size =5><strong>性质</strong></font></p>
<p>1、非负性</p>
<p>2、常数随机变量的方差为0</p>
<p>3、<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>a</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span></span></span></span>，其中<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span>为常数</p>
<p>4、<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>Var</span></span></span><span>(</span><span>a</span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.0641em;vertical-align:-0.25em;"></span><span><span>a</span><span><span><span><span style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span></span></span></span></p>
<p>5、两个随机变量的方差：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:4.5241em;vertical-align:-2.0121em;"></span><span><span><span><span><span style="height:2.5121em;"><span style="top:-4.5121em;"><span style="height:2.8641em;"></span><span></span></span><span style="top:-3.0121em;"><span style="height:2.8641em;"></span><span></span></span><span style="top:-1.5121em;"><span style="height:2.8641em;"></span><span></span></span></span><span>​</span></span><span><span style="height:2.0121em;"><span></span></span></span></span></span><span><span><span><span style="height:2.5121em;"><span style="top:-4.6479em;"><span style="height:3em;"></span><span><span></span><span><span><span>Var</span><span>(</span><span>aX</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span style="margin-right:0.025em;">bY</span><span>)</span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span>a</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span><span>b</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.22222em;">Y</span><span>)</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span>2</span><span>ab</span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span><span>)</span></span></span><span style="top:-3.1479em;"><span style="height:3em;"></span><span><span></span><span><span><span>Var</span><span>(</span><span>X</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span style="margin-right:0.025em;">Y</span><span>)</span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.22222em;">Y</span><span>)</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span>2</span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span><span>)</span></span></span><span style="top:-1.6479em;"><span style="height:3em;"></span><span><span></span><span>s</span><span>.</span><span>t</span><span>.</span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span style="margin-right:0.05764em;">E</span><span>[(</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span><span>μ</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span>(</span><span style="margin-right:0.22222em;">Y</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span><span>μ</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.22222em;">Y</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)]</span><span> </span><span>代表协方差</span></span></span></span><span>​</span></span><span><span style="height:2.0121em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:2.5121em;"><span style="top:-4.5121em;"><span style="height:2.8641em;"></span><span></span></span><span style="top:-3.0121em;"><span style="height:2.8641em;"></span><span></span></span><span style="top:-1.5121em;"><span style="height:2.8641em;"></span><span></span></span></span><span>​</span></span><span><span style="height:2.0121em;"><span></span></span></span></span></span></span></span></span></p>
<p>6、对于<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">N</span></span></span></span>个随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.10903em;">N</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>}</span></span></span></span>的总和：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:3.106em;vertical-align:-1.2777em;"></span><span><span><span>Var</span></span></span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;"><span>(</span></span><span><span><span><span style="height:1.8283em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span style="margin-right:0.10903em;">N</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="top:0em;"><span>)</span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:3.2421em;vertical-align:-1.4138em;"></span><span><span><span><span style="height:1.8283em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>,</span><span style="margin-right:0.05724em;">j</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span style="margin-right:0.10903em;">N</span></span></span></span><span>​</span></span><span><span style="height:1.4138em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.22222em;">Y</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.2222em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:3.106em;vertical-align:-1.2777em;"></span><span><span><span><span style="height:1.8283em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span style="margin-right:0.10903em;">N</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span><span>Var</span></span></span><span>(</span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:2.4882em;vertical-align:-1.4382em;"></span><span><span><span><span style="height:1.05em;"><span style="top:-1.8479em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span><span><span><span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span><span></span></span></span><span></span></span></span></span></span><span>=</span></span><span style="margin-right:0.05724em;">j</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span></span><span>​</span></span><span><span style="height:1.4382em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.22222em;">Y</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.2222em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>)</span></span></span></span></span></p>
<h2 id="协方差-covariance"> 协方差（covariance）</h2>
<p>用于衡量两个随机变量的联合变化程度。</p>
<p>方差（variance）是协方差一种特殊情况，即该变量与自身的协方差。</p>
<p><font size =5><strong>定义</strong></font></p>
<p>期望值分别为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span>μ</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>与<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>(</span><span style="margin-right:0.22222em;">Y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span>μ</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.22222em;">Y</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>的两个具有有限<strong>二阶矩</strong>的实数随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span></span></span></span>之间的协方差为：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[(</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>μ</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span>(</span><span style="margin-right:0.22222em;">Y</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>μ</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.22222em;">Y</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)]</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>[</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>⋅</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">Y</span><span>]</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span>μ</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>μ</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.22222em;">Y</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span></span></p>
<p>如果<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span></span></span></span>是相互独立的，则两者之间的协方差为0，反之不成立。</p>
<p>如果<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span></span></span></span>两个变量的变化趋势相同，协方差为正值；变化趋势相反，协方差为负值。</p>
<p><font size =5><strong>相关系数</strong></font></p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>ρ</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.557em;vertical-align:-1.13em;"></span><span><span></span><span><span><span><span style="height:1.427em;"><span style="top:-2.175em;"><span style="height:3em;"></span><span><span><span><span><span style="height:0.935em;"><span style="top:-3.2em;"><span style="height:3.2em;"></span><span style="padding-left:1em;"><span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span></span><span style="margin-right:0.2222em;"></span><span>⋅</span><span style="margin-right:0.2222em;"></span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.22222em;">Y</span><span>)</span></span></span><span style="top:-2.895em;"><span style="height:3.2em;"></span><span style="min-width:1.02em;height:1.28em;"><svg xmlns="http://www.w3.org/2000/svg" width='400em' height='1.28em' viewBox='0 0 400000 1296' preserveAspectRatio='xMinYMin slice'><path d='M263,681c0.7,0,18,39.7,52,119
c34,79.3,68.167,158.7,102.5,238c34.3,79.3,51.8,119.3,52.5,120
c340,-704.7,510.7,-1060.3,512,-1067
l0 -0
c4.7,-7.3,11,-11,19,-11
H40000v40H1012.3
s-271.3,567,-271.3,567c-38.7,80.7,-84,175,-136,283c-52,108,-89.167,185.3,-111.5,232
c-22.3,46.7,-33.8,70.3,-34.5,71c-4.7,4.7,-12.3,7,-23,7s-12,-1,-12,-1
s-109,-253,-109,-253c-72.7,-168,-109.3,-252,-110,-252c-10.7,8,-22,16.7,-34,26
c-22,17.3,-33.3,26,-34,26s-26,-26,-26,-26s76,-59,76,-59s76,-60,76,-60z
M1001 80h400000v40h-400000z'/></svg></span></span></span><span>​</span></span><span><span style="height:0.305em;"><span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span><span>)</span></span></span></span><span>​</span></span><span><span style="height:1.13em;"><span></span></span></span></span></span><span></span></span></span></span></span></span></p>
<p>其中，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7335em;vertical-align:-0.1944em;"></span><span>ρ</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[</span><span>−</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>1</span><span>]</span></span></span></span>，当<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>ρ</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>1</span></span></span></span>是完全线性相关，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>ρ</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7278em;vertical-align:-0.0833em;"></span><span>−</span><span>1</span></span></span></span>是完全线性负相关。</p>
<p>当<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>ρ</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span>时，协方差为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6444em;"></span><span>0</span></span></span></span>，两个随机变量称为线性不相关。</p>
<p><font size =5><strong>性质</strong></font></p>
<p>如果<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span></span></span></span>是实数随机变量，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span>a</span><span>,</span><span style="margin-right:0.1667em;"></span><span>b</span></span></span></span>是实数，</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:4.5em;vertical-align:-2em;"></span><span><span><span><span><span style="height:2.5em;"><span style="top:-4.5em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-3em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-1.5em;"><span style="height:2.84em;"></span><span></span></span></span><span>​</span></span><span><span style="height:2em;"><span></span></span></span></span></span><span><span><span><span style="height:2.5em;"><span style="top:-4.66em;"><span style="height:3em;"></span><span><span></span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span>Var</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span></span></span><span style="top:-3.16em;"><span style="height:3em;"></span><span><span></span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span style="margin-right:0.22222em;">Y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.07847em;">X</span><span>)</span></span></span><span style="top:-1.66em;"><span style="height:3em;"></span><span><span></span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span>a</span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">bY</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>ab</span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span style="margin-right:0.22222em;">Y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.07847em;">X</span><span>)</span></span></span></span><span>​</span></span><span><span style="height:2em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:2.5em;"><span style="top:-4.5em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-3em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-1.5em;"><span style="height:2.84em;"></span><span></span></span></span><span>​</span></span><span><span style="height:2em;"><span></span></span></span></span></span></span></span></span></p>
<p>对于随机变量序列<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>与$ Y_1,Y_2,\dots,Y_n$，有：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:3.1638em;vertical-align:-1.4138em;"></span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;"><span>(</span></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span style="margin-right:0.05724em;">j</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.4138em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.22222em;">Y</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.2222em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span style="top:0em;"><span>)</span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:3.0652em;vertical-align:-1.4138em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span style="margin-right:0.05724em;">j</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:1.4138em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.22222em;">Y</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.2222em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>)</span></span></span></span></span></p>
<p>对于随机变量序列<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>，有：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:3.0277em;vertical-align:-1.2777em;"></span><span><span><span>Var</span></span></span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;"><span>(</span></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="top:0em;"><span>)</span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.9291em;vertical-align:-1.2777em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span><span>Var</span></span></span><span>(</span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:2.4638em;vertical-align:-1.4138em;"></span><span>2</span><span style="margin-right:0.1667em;"></span><span><span><span><span style="height:1.05em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>,</span><span style="margin-right:0.05724em;">j</span><span>:</span><span>i</span><span>&lt;</span><span style="margin-right:0.05724em;">j</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span></span><span>​</span></span><span><span style="height:1.4138em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>)</span></span></span></span></span></p>
<h2 id="协方差矩阵"> 协方差矩阵</h2>
<p>协方差矩阵的每个元素是各个向量元素之间的协方差，是从标量随机变量到高维度随机向量的自然推广。</p>
<p>设<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.0913em;vertical-align:-0.25em;"></span><span>(</span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.10903em;">N</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>)</span><span><span><span><span style="height:0.8413em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.13889em;">T</span></span></span></span></span></span></span></span></span></span></span>为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>维随机变量，矩阵：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.0361em;vertical-align:-0.2861em;"></span><span>(</span><span><span>c</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span style="margin-right:0.05724em;">ij</span></span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span><span>)</span><span><span><span><span style="height:0.2583em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>n</span><span>×</span><span>n</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:5.46em;vertical-align:-2.48em;"></span><span><span><span><span><span><span style="height:2.95em;"><span style="top:-2.011em;"><span style="height:3.816em;"></span><span><span>⎝</span></span></span><span style="top:-3.158em;"><span style="height:3.816em;"></span><span style="height:1.816em;width:0.875em;"><svg xmlns="http://www.w3.org/2000/svg" width='0.875em' height='1.816em' style='width:0.875em' viewBox='0 0 875 1816' preserveAspectRatio='xMinYMin'><path d='M291 0 H417 V1816 H291z M291 0 H417 V1816 H291z'/></svg></span></span><span style="top:-5.6111em;"><span style="height:3.816em;"></span><span><span>⎛</span></span></span></span><span>​</span></span><span><span style="height:2.45em;"><span></span></span></span></span></span></span><span><span><span><span><span><span style="height:2.98em;"><span style="top:-5.8275em;"><span style="height:3.6875em;"></span><span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>11</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-4.6275em;"><span style="height:3.6875em;"></span><span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>21</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-2.7675em;"><span style="height:3.6875em;"></span><span><span><span>⋮</span><span style="border-right-width:0em;border-top-width:1.5em;bottom:0em;"></span></span></span></span><span style="top:-1.5675em;"><span style="height:3.6875em;"></span><span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>n</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:2.48em;"><span></span></span></span></span></span><span style="width:0.5em;"></span><span style="width:0.5em;"></span><span><span><span><span style="height:2.98em;"><span style="top:-5.8275em;"><span style="height:3.6875em;"></span><span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>12</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-4.6275em;"><span style="height:3.6875em;"></span><span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>22</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-2.7675em;"><span style="height:3.6875em;"></span><span><span><span>⋮</span><span style="border-right-width:0em;border-top-width:1.5em;bottom:0em;"></span></span></span></span><span style="top:-1.5675em;"><span style="height:3.6875em;"></span><span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>n</span><span>2</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:2.48em;"><span></span></span></span></span></span><span style="width:0.5em;"></span><span style="width:0.5em;"></span><span><span><span><span style="height:2.98em;"><span style="top:-5.64em;"><span style="height:3.5em;"></span><span><span>…</span></span></span><span style="top:-4.44em;"><span style="height:3.5em;"></span><span><span>…</span></span></span><span style="top:-2.58em;"><span style="height:3.5em;"></span><span><span>⋱</span></span></span><span style="top:-1.38em;"><span style="height:3.5em;"></span><span><span>…</span></span></span></span><span>​</span></span><span><span style="height:2.48em;"><span></span></span></span></span></span><span style="width:0.5em;"></span><span style="width:0.5em;"></span><span><span><span><span style="height:2.98em;"><span style="top:-5.8275em;"><span style="height:3.6875em;"></span><span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>1</span><span>n</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-4.6275em;"><span style="height:3.6875em;"></span><span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>2</span><span>n</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-2.7675em;"><span style="height:3.6875em;"></span><span><span><span>⋮</span><span style="border-right-width:0em;border-top-width:1.5em;bottom:0em;"></span></span></span></span><span style="top:-1.5675em;"><span style="height:3.6875em;"></span><span><span><span>c</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>nn</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:2.48em;"><span></span></span></span></span></span></span></span><span><span><span><span><span style="height:2.95em;"><span style="top:-2.011em;"><span style="height:3.816em;"></span><span><span>⎠</span></span></span><span style="top:-3.158em;"><span style="height:3.816em;"></span><span style="height:1.816em;width:0.875em;"><svg xmlns="http://www.w3.org/2000/svg" width='0.875em' height='1.816em' style='width:0.875em' viewBox='0 0 875 1816' preserveAspectRatio='xMinYMin'><path d='M457 0 H583 V1816 H457z M457 0 H583 V1816 H457z'/></svg></span></span><span style="top:-5.6111em;"><span style="height:3.816em;"></span><span><span>⎞</span></span></span></span><span>​</span></span><span><span style="height:2.45em;"><span></span></span></span></span></span></span></span></span></span></span></span></p>
<p>为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>维随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>的协方差矩阵，其中：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7167em;vertical-align:-0.2861em;"></span><span><span>c</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span style="margin-right:0.05724em;">ij</span></span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.0361em;vertical-align:-0.2861em;"></span><span><span><span style="margin-right:0.01389em;">Cov</span></span></span><span>(</span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>)</span><span>;</span><span> </span><span style="margin-right:0.1667em;"></span><span>i</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.05724em;">j</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8389em;vertical-align:-0.1944em;"></span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span>n</span></span></span></span></span></p>
<p>为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>的分量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.9694em;vertical-align:-0.2861em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span></span></span></span>的协方差（假设都存在）。</p>
<p>由于<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7167em;vertical-align:-0.2861em;"></span><span><span>c</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span style="margin-right:0.05724em;">ij</span></span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.0361em;vertical-align:-0.2861em;"></span><span><span>c</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>ji</span></span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>(</span><span>i</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.05724em;">j</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span>n</span><span>)</span></span></span></span>，因为协方差矩阵是对称<strong>非负正定矩阵</strong>（特征值全为正数）</p>
<h2 id="中心矩-central-moment"> 中心矩（central moment）</h2>
<p>某一个随机变量的平均值构成其概率分布的矩（也叫动差），高阶中心矩仅与分布和形状相关，与分布的位置无关。</p>
<p><font size =5><strong>定义</strong></font></p>
<p>对于一维随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>，其<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span style="margin-right:0.03148em;">k</span></span></span></span>阶中心距<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span>μ</span><span><span><span><span style="height:0.3361em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.03148em;">k</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>是相对于<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>的期望的<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span style="margin-right:0.03148em;">k</span></span></span></span>阶矩：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span>μ</span><span><span><span><span style="height:0.3361em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.03148em;">k</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.05764em;">E</span><span>[(</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.1491em;vertical-align:-0.25em;"></span><span style="margin-right:0.05764em;">E</span><span>[</span><span style="margin-right:0.07847em;">X</span><span>]</span><span><span>)</span><span><span><span><span style="height:0.8991em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.03148em;">k</span></span></span></span></span></span></span></span><span>]</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.3846em;vertical-align:-0.9703em;"></span><span><span style="margin-right:0.44445em;position:relative;top:-0.0011em;">∫</span><span><span><span><span style="height:1.4143em;"><span style="top:-1.7881em;margin-left:-0.4445em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>−</span><span>∞</span></span></span></span><span style="top:-3.8129em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>∞</span></span></span></span></span><span>​</span></span><span><span style="height:0.9703em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.1491em;vertical-align:-0.25em;"></span><span>μ</span><span><span>)</span><span><span><span><span style="height:0.8991em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.03148em;">k</span></span></span></span></span></span></span></span><span style="margin-right:0.10764em;">f</span><span>(</span><span>x</span><span>)</span><span>d</span><span>x</span></span></span></span></span></p>
<p>其中：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:7.5em;vertical-align:-3.5em;"></span><span><span><span><span><span style="height:4em;"><span style="top:-6em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-4.5em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-3em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-1.5em;"><span style="height:2.84em;"></span><span></span></span><span style="top:0em;"><span style="height:2.84em;"></span><span></span></span></span><span>​</span></span><span><span style="height:3.5em;"><span></span></span></span></span></span><span><span><span><span style="height:4em;"><span style="top:-6.16em;"><span style="height:3em;"></span><span><span></span><span><span>μ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>0</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>1</span></span></span><span style="top:-4.66em;"><span style="height:3em;"></span><span><span></span><span><span>μ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>0</span></span></span><span style="top:-3.16em;"><span style="height:3em;"></span><span><span></span><span><span>μ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>Va</span><span style="margin-right:0.02778em;">r</span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span><span> </span><span>方差</span></span></span><span style="top:-1.66em;"><span style="height:3em;"></span><span><span></span><span><span>μ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>3</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>定义为</span><span style="margin-right:0.07847em;">X</span><span>的偏度</span></span></span><span style="top:-0.16em;"><span style="height:3em;"></span><span><span></span><span><span>μ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>4</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>定义为</span><span style="margin-right:0.07847em;">X</span><span>的峰度</span></span></span></span><span>​</span></span><span><span style="height:3.5em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:4em;"><span style="top:-6em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-4.5em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-3em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-1.5em;"><span style="height:2.84em;"></span><span></span></span><span style="top:0em;"><span style="height:2.84em;"></span><span></span></span></span><span>​</span></span><span><span style="height:3.5em;"><span></span></span></span></span></span></span></span></span></p>
<p><font size =5><strong>性质</strong></font></p>
<p>1、中心矩具有平移不变性：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>μ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>c</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>μ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span></span></span></span></span></p>
<p>2、<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>阶中心矩是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>次齐次函数：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>μ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>c</span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>c</span><span><span><span><span style="height:0.7144em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span></span></span></span></span><span><span>μ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span></span></span></span></span></p>
<p>3、只有当<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span>n</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>3</span><span>}</span></span></span></span>，且<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span></span></span></span>是两个相互独立的随机变量时，中心矩具有加法性：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>μ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">Y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>μ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>μ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span style="margin-right:0.22222em;">Y</span><span>)</span></span></span></span></span></p>
]]></content:encoded>
    </item>
    <item>
      <title>不完备信息系统-粗糙集理论</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/mathematics/%E7%B2%97%E7%B3%99%E9%9B%86%E7%90%86%E8%AE%BA/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/mathematics/%E7%B2%97%E7%B3%99%E9%9B%86%E7%90%86%E8%AE%BA/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">不完备信息系统-粗糙集理论</source>
      <description>利用粗糙集（rough set）理论，解决不完备信息系统（Incomplete Information System, IIS）下的特征关系表示。</description>
      <category>Maths</category>
      <pubDate>Sun, 22 May 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="信息系统-information-system"> 信息系统（Information system）</h2>
<p>In Rough Set, data model information is stored in a table. Each row(tuples)(行/元组)represents a fact or an object. Often the facts are not consistent(符合，一致) with each other. In Rough Set terminology(术语), a data table is called an information system(IS). Thus, the information table represents input data, gathered from any domain.</p>
<table>
<thead>
<tr>
<th></th>
<th>attributes</th>
<th>attributes</th>
<th>attributes</th>
<th>attributes</th>
</tr>
</thead>
<tbody>
<tr>
<td>case</td>
<td>Temperature</td>
<td>Headache</td>
<td>Nausea(恶心)</td>
<td>Cough</td>
</tr>
<tr>
<td>1</td>
<td>high</td>
<td>yes</td>
<td>no</td>
<td>yes</td>
</tr>
<tr>
<td>2</td>
<td>very high</td>
<td>yes</td>
<td>yes</td>
<td>no</td>
</tr>
<tr>
<td>3</td>
<td>normal</td>
<td>no</td>
<td>no</td>
<td>yes</td>
</tr>
</tbody>
</table>
<p>Note: Rows of a table are called examples(objects, entities(实体)).</p>
<p>Formally, an information system(IS) is pair<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span style="margin-right:0.10903em;">U</span><span>,</span><span style="margin-right:0.1667em;"></span><span>A</span><span style="margin-right:0.13889em;">T</span><span>)</span></span></span></span>, and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">U</span></span></span></span> is a non-empty finite(有限的) set of objects, called the universe(论域), and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span><span style="margin-right:0.13889em;">T</span></span></span></span> is a non-empty finite set of attributes(属性). If some attribute values are unknown, the IS is IIS. In IIS, the unknown attribute value is denoted by(用...表示) &quot;<code>*</code>&quot;.</p>
<h2 id="不可分辨关系-indiscernibility-relation"> 不可分辨关系（indiscernibility relation）</h2>
<p>Each subset of attributes <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8193em;vertical-align:-0.136em;"></span><span>A</span><span style="margin-right:0.2778em;"></span><span>⊆</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span>A</span><span style="margin-right:0.13889em;">T</span></span></span></span> determines a binary indiscernibility relation <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>IND</span></span></span><span>(</span><span>A</span><span>)</span></span></span></span> and is denoted by <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10903em;">U</span><span>/</span><span><span><span>IND</span></span></span><span>(</span><span>A</span><span>)</span></span></span></span>or <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span style="margin-right:0.10903em;">U</span><span>/</span><span>A</span><span>)</span></span></span></span>, as follows:</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>IND</span></span></span><span>(</span><span>A</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7667em;vertical-align:-0.0833em;"></span><span style="margin-right:0.10903em;">U</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10903em;">U</span><span>∣∀</span><span>a</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>A</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>a</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>)}</span><span>.</span></span></span></span></span></p>
<p>The relation <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>IND</span></span></span><span>(</span><span>A</span><span>)</span></span></span></span> is called an <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7667em;vertical-align:-0.0833em;"></span><span>A</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span>in</span><span>d</span><span>i</span><span style="margin-right:0.02778em;">scer</span><span>nibi</span><span style="margin-right:0.01968em;">l</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span> </span><span>re</span><span style="margin-right:0.01968em;">l</span><span>a</span><span>t</span><span>i</span><span>o</span><span>n</span></span></span></span>.</p>
<p>If <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>IND</span></span></span><span>(</span><span>A</span><span>)</span></span></span></span>， then <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span> and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span style="margin-right:0.03588em;">y</span></span></span></span> are indiscernible(or indistinguishable) by attributes from <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>.</p>
<p>The indiscernibility relation is an <strong>equivalence relation</strong>(等价关系). Sets that are indiscernibility are called elementary sets(基本集/商集).</p>
<p>不可分辨关系也称为等价关系。</p>
<p>The equivalence classes of the <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.13889em;">P</span></span></span></span>-indiscernibility relation are denoted by <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[</span><span>x</span><span><span>]</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>.</p>
<table>
<thead>
<tr>
<th>Object</th>
<th>p1</th>
<th>p2</th>
<th>p3</th>
<th>p4</th>
<th>p5</th>
</tr>
</thead>
<tbody>
<tr>
<td>O1</td>
<td>1</td>
<td>2</td>
<td>0</td>
<td>1</td>
<td>1</td>
</tr>
<tr>
<td>O2</td>
<td>1</td>
<td>2</td>
<td>0</td>
<td>1</td>
<td>1</td>
</tr>
<tr>
<td>O3</td>
<td>2</td>
<td>0</td>
<td>0</td>
<td>1</td>
<td>0</td>
</tr>
<tr>
<td>O4</td>
<td>0</td>
<td>0</td>
<td>1</td>
<td>2</td>
<td>1</td>
</tr>
<tr>
<td>O5</td>
<td>2</td>
<td>1</td>
<td>0</td>
<td>2</td>
<td>1</td>
</tr>
<tr>
<td>O6</td>
<td>0</td>
<td>0</td>
<td>1</td>
<td>2</td>
<td>2</td>
</tr>
<tr>
<td>O7</td>
<td>2</td>
<td>0</td>
<td>0</td>
<td>1</td>
<td>0</td>
</tr>
<tr>
<td>O8</td>
<td>0</td>
<td>1</td>
<td>2</td>
<td>2</td>
<td>1</td>
</tr>
<tr>
<td>O9</td>
<td>2</td>
<td>1</td>
<td>0</td>
<td>2</td>
<td>2</td>
</tr>
<tr>
<td>O10</td>
<td>2</td>
<td>0</td>
<td>0</td>
<td>1</td>
<td>0</td>
</tr>
</tbody>
</table>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>INA</span><span>({</span><span>p1</span><span>})</span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{{</span><span style="margin-right:0.02778em;">O</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>2</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>5</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>7</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>9</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>10</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>8</span><span>}}</span></span></span></span></span></p>
<p>In the above example,</p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>2</span><span>}</span></span></span></span> is an <strong>equivalence class(等价类)</strong>.</p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.02778em;">O</span><span>1</span></span></span></span> and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.02778em;">O</span><span>2</span></span></span></span> are characterized by the same values of attribute <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8389em;vertical-align:-0.1944em;"></span><span>p</span><span>1</span></span></span></span> and the value is <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6444em;"></span><span>1</span></span></span></span>.</p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.02778em;">O</span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>5</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>7</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>9</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>10</span></span></span></span> are characterized by the same value of attribute <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8389em;vertical-align:-0.1944em;"></span><span>p</span><span>1</span></span></span></span> and the value is <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6444em;"></span><span>2</span></span></span></span>.</p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.02778em;">O</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>8</span></span></span></span> are characterized by the same value of attribute <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8389em;vertical-align:-0.1944em;"></span><span>p</span><span>1</span></span></span></span> and the value is <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6444em;"></span><span>0</span></span></span></span>.</p>
<h2 id="近似-approximations"> 近似（approximations）</h2>
<p>It is a formal approximation of a crisp set(明确的集合) defined by its two approximations- <strong>Upper approximation</strong> and <strong>Lower approximation</strong>.</p>
<p>Let <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8193em;vertical-align:-0.136em;"></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>⊆</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">U</span></span></span></span> be a target set that we wish to represent using attribute subset <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span>A</span><span>,</span><span style="margin-right:0.1667em;"></span><span>A</span><span style="margin-right:0.2778em;"></span><span>⊆</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span>A</span><span style="margin-right:0.13889em;">T</span></span></span></span>.</p>
<ul>
<li>Lower approximation: 被包含在<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>里面的等价关系的并集</li>
</ul>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8833em;vertical-align:-0.2em;"></span><span><span><span><span style="height:0.6833em;"><span style="top:-2.84em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.2em;"><span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span></span></span></span> is a lower approximation of <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>, iff (当且仅当)</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8833em;vertical-align:-0.2em;"></span><span><span><span><span style="height:0.6833em;"><span style="top:-2.84em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.2em;"><span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>∪</span><span>{[</span><span>x</span><span><span>]</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span> </span><span>∣</span><span>x</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10903em;">U</span><span>,</span><span> </span><span style="margin-right:0.1667em;"></span><span>[</span><span>x</span><span><span>]</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>⊆</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.07847em;">X</span><span>}</span></span></span></span></span></p>
<ul>
<li>Upper approximation: 与<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>有交的等价关系的并集</li>
</ul>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8201em;"></span><span><span><span><span style="height:0.8201em;"><span style="top:-3em;"><span style="height:3em;"></span><span>A</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.1111em;"><span>ˉ</span></span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span></span></span></span> is an upper approximation of <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>, iff</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8201em;"></span><span><span><span><span style="height:0.8201em;"><span style="top:-3em;"><span style="height:3em;"></span><span>A</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.1111em;"><span>ˉ</span></span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>∪</span><span>{[</span><span>x</span><span><span>]</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span> </span><span>∣</span><span>x</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10903em;">U</span><span>,</span><span> </span><span style="margin-right:0.1667em;"></span><span>[</span><span>x</span><span><span>]</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span><span></span></span></span><span></span></span></span></span></span><span>=</span></span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>∅</span><span>}</span></span></span></span></span></p>
<p>The set <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7667em;vertical-align:-0.0833em;"></span><span style="margin-right:0.10903em;">U</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8201em;"></span><span><span><span><span style="height:0.8201em;"><span style="top:-3em;"><span style="height:3em;"></span><span>A</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.1111em;"><span>ˉ</span></span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span></span></span></span> therefore represents the <strong>negative region(负域)</strong>, containing the set of objects that can be definitely ruled out as members of the target set <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>.</p>
<center><img src=".\images\boundary-region.png" width=50%></center>
<p><strong>Boundary region</strong></p>
<p>The boundary region, given by set difference <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.9034em;vertical-align:-0.0833em;"></span><span><span><span><span style="height:0.8201em;"><span style="top:-3em;"><span style="height:3em;"></span><span>A</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.1111em;"><span>ˉ</span></span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8833em;vertical-align:-0.2em;"></span><span><span><span><span style="height:0.6833em;"><span style="top:-2.84em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.2em;"><span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span></span></span></span>, consists of those objects that can neither be ruled in nor ruled out as members of the target set <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>.【差集：在<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8201em;"></span><span><span><span><span style="height:0.8201em;"><span style="top:-3em;"><span style="height:3em;"></span><span>A</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.1111em;"><span>ˉ</span></span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span></span></span></span>中的元素，但不在<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8833em;vertical-align:-0.2em;"></span><span><span><span><span style="height:0.6833em;"><span style="top:-2.84em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.2em;"><span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span></span></span></span>中的元素】</p>
<h2 id="the-rough-set-粗糙集"> The rough set（粗糙集）</h2>
<p>The tuple <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.0701em;vertical-align:-0.25em;"></span><span>⟨</span><span><span><span><span style="height:0.8201em;"><span style="top:-3em;"><span style="height:3em;"></span><span>A</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.1111em;"><span>ˉ</span></span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span><span><span><span style="height:0.6833em;"><span style="top:-2.84em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.2em;"><span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span><span>⟩</span></span></span></span> composed of the lower and upper approximation is called a rough set.</p>
<p>A set is said to be rough if its boundary region is non-empty, otherwise the set is crisp set(明确集).</p>
<div><p>提示</p>
<p>Target set: <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>({</span><span style="margin-right:0.02778em;">O</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>5</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>8</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>7</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>9</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>10</span><span>})</span></span></span></span></p>
<p>Attributes subset: <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span>p</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>p</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>p</span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span>p</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>p</span><span>5</span><span>}</span></span></span></span></p>
<p>不可分辨关系：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>IND</span></span></span><span>(</span><span>A</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>({</span><span style="margin-right:0.02778em;">O</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>2</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>7</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>10</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>4</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>5</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>6</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>8</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>9</span><span>})</span></span></span></span></p>
<p>基本集<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[</span><span>x</span><span><span>]</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>2</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>7</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>10</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>4</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>5</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>6</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>8</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>9</span><span>}</span></span></span></span></p>
<p>基本集和目标集<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>关系</p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8833em;vertical-align:-0.2em;"></span><span><span><span><span style="height:0.6833em;"><span style="top:-2.84em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.2em;"><span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>∪</span><span>{</span><span>x</span><span>∣</span><span> </span><span>[</span><span>x</span><span><span>]</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>⊆</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.07847em;">X</span><span>}</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>5</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>8</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>9</span><span>}</span></span></span></span></p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8201em;"></span><span><span><span><span style="height:0.8201em;"><span style="top:-3em;"><span style="height:3em;"></span><span>A</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.1111em;"><span>ˉ</span></span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>∪</span><span>{</span><span>x</span><span>∣</span><span> </span><span>[</span><span>x</span><span><span>]</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span><span></span></span></span><span></span></span></span></span></span><span>=</span></span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>∅</span><span>}</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>7</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>10</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>5</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>8</span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>7</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>9</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>10</span><span>}</span></span></span></span></p>
<p>Positive region(正域) = <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8833em;vertical-align:-0.2em;"></span><span>∪</span><span><span><span><span style="height:0.6833em;"><span style="top:-2.84em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.2em;"><span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span></span></span></span> =<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>5</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>8</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>9</span><span>}</span></span></span></span></p>
<p>Negative region(负域) = <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7667em;vertical-align:-0.0833em;"></span><span style="margin-right:0.10903em;">U</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8201em;"></span><span>∪</span><span><span><span><span style="height:0.8201em;"><span style="top:-3em;"><span style="height:3em;"></span><span>A</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.1111em;"><span>ˉ</span></span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6944em;"></span><span>n</span><span>u</span><span style="margin-right:0.01968em;">ll</span></span></span></span></p>
<p>Boundary region = <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.9034em;vertical-align:-0.0833em;"></span><span><span><span><span style="height:0.8201em;"><span style="top:-3em;"><span style="height:3em;"></span><span>A</span></span><span style="top:-3.2523em;"><span style="height:3em;"></span><span style="left:-0.1111em;"><span>ˉ</span></span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8833em;vertical-align:-0.2em;"></span><span><span><span><span style="height:0.6833em;"><span style="top:-2.84em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.2em;"><span></span></span></span></span></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span style="margin-right:0.02778em;">O</span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>7</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02778em;">O</span><span>10</span><span>}</span></span></span></span>，即在前者中而不在后者中的元素。</p>
<p>所以<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>是粗糙集。</p>
</div>
<p>The notion(概念) of Rough sets was introduced by Z Pawlak in his seminal paper of 1982<sup></sup>. It is a formal theory derived from fundamental research on the logical properties of information systems. Rough set theory has been a methodology(方法论) for database mining or knowledge discovery in relational databases. We can use a rough set approach to discover structural relationships within imprecise and noisy data<sup></sup>.</p>
<p><strong>Basic problems in data analysis solved by Rough Set:</strong></p>
<ul>
<li>Characterization(表征，描述) of a set of objects in terms of (在...方面)attribute values.</li>
<li>Finding dependency(依赖关系) between the attributes.</li>
<li>Reduction of superfluous(多余的) attributes.</li>
<li>Finding the most significant attributes.</li>
<li>Decision rule generation(生成决策规则).</li>
</ul>
<p><strong>Goals of Rough Set Theory:</strong></p>
<ul>
<li>
<p>The main goal of the rough set analysis is the induction(归纳，导致，产生) of (learning) approximations(近似) of concepts. Rough sets constitute(构成，组成) a sound(合理可靠的) basis of <strong>KDD</strong>(知识发现-Knowledge Discovery in Database). It offers mathematical tools to discover patterns hidden in data.</p>
</li>
<li>
<p>It can be used for feature selection, feature extraction, data reduction(数据简化), decision rule generation, and pattern extraction(templates, association rules(关联规则)) etc.</p>
</li>
<li>
<p>Identifies partial(部分) or total dependencies in data, eliminates(消除) redundant(冗余的) data, and gives an approach to null values, missing data, dynamic data and others.</p>
</li>
</ul>
<h2 id="知识发现-knowledge-discovery-in-database"> 知识发现（knowledge discovery in database）</h2>
<p>KDD is the process of discovering useful knowledge from a collection of data. This widely used data mining technique is a process that includes data preparation and selection, data cleansing(数据清洗), incorporating(整合) prior knowledge of data sets and interpreting accurate solutions from the observed results(根据观测结果准确解释).</p>
<p>The KDD process has reached its peak(顶峰) in the last 10 years. It now houses(贮藏) many different approaches to discovery, which include inductive learning(归纳学习), Bayesian statistics, semantic query optimization(语义查询优化), knowledge acquisition for expert systems and information theory. The ultimate(最终) goal is to extract <strong>high-level</strong> knowledge from <strong>low-level data</strong>.</p>
<blockquote>
<p>high-level describes those operations that are more abstract in nature</p>
<p>low-level describes more specific individual components of a systematic operation.</p>
</blockquote>
<h2 id="不完备信息系统-iis"> 不完备信息系统（IIS）</h2>
<p>In an incomplete information system (IIS), some attribute values for objects may be unknown (missing, null). But in practice, because of errors in data measuring, the limitations of acquiring data, some human factors, etc. IIS often occur in knowledge acquisition.</p>
<p>There are usually two methods in <strong>rough set theory</strong>（粗糙集理论） to deal with IIS: (1) <strong>data reparation</strong>（数据修复）and (2) <strong>model extension</strong><sup></sup>.  The model extension（模型扩展） is more suitable than data reparation for processing IIS, which does not change the original information of IIS and is more objective for the knowledge system generation.</p>
<h2 id="广义不可分辨关系-generalized-indiscernibility-relations"> 广义不可分辨关系（generalized indiscernibility relations）</h2>
<p>At present, various extension models of the classical rough set theory have been proposed, in which the indiscernibility relation was extended to some <strong>non-equivalence relations</strong>(非等价关系), or called generalized indiscernibility relations, such as <strong>tolerance relation(容差关系)<sup></sup>, non-symmetric similarity relation(非对称相似关系)<sup></sup>, limited tolerance relation(有限容差关系)<sup></sup>, valued tolerance relation(量化容差关系)<sup></sup>, and characteristic relation(特征关系)<sup></sup></strong>.</p>
<p>These extension models have their own limitations<sup></sup>.</p>
<p>In tolerance relation, two objects that have no one known and the same value on all attributes may be considered as indiscernible(不可分的) and classified in the same class.(适合处理未知属性均为遗漏的)</p>
<p>In non-symmetric similarity relation, two objects that have a lot of known and same values on all attributes may be separated, and classified into different classes.(适合处理未知属性均为丢失的)</p>
<p>The limited tolerance relation inherits(继承) the merit(优点) of the tolerance relation and non-symmetric similarity relation and avoids their limitations. But, in limited tolerance relation, two objects may be considered similar if only they have one known and the same value on all attributes. This condition is too lenient(宽泛的，不严谨的) for the IIS with a large number of attributes.(针对具有大量属性的不完备信息系统而言，限制性容差关系条件过于宽泛)</p>
<p>The characteristic relation, which is a generalization of both tolerance relation and non-symmetric similarity relation, can not avoid their limitations.</p>
<h2 id="容差关系-tolerance-relation"> 容差关系（tolerance relation）</h2>
<p>Let <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>SIM</span></span></span><span>(</span><span>A</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>A</span><span style="margin-right:0.2778em;"></span><span>⊆</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span>A</span><span style="margin-right:0.13889em;">T</span></span></span></span>, denote binary similarity relation between objects that are possibly indiscernible in terms of(在...方面) values of attributes <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span></span></span></span> (i.e. we cannot say with certainty(确定的说) that these objects are different).</p>
<p>In general, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>SIM</span></span></span><span>(</span><span>A</span><span>)</span></span></span></span> could be any relation between objects that we want to treat as indiscernible. Let us define similarity relation more precisely:</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>SIM</span></span></span><span>(</span><span>A</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7667em;vertical-align:-0.0833em;"></span><span style="margin-right:0.10903em;">U</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10903em;">U</span><span>∣∀</span><span>a</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>A</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>a</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>)</span><span> </span><span><span><span>or</span></span></span><span> </span><span>a</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>∗</span><span> </span><span><span><span>or</span></span></span><span> </span><span>a</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>∗</span><span>}</span><span>.</span></span></span></span></span></p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span><span>SIM</span></span></span></span><span>(</span><span>A</span><span>)</span></span></span></span>is a <strong>tolerance relation(容差关系)</strong>:<sup></sup></p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>SIM</span></span></span><span>(</span><span>A</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>∩</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>a</span><span>∈</span><span>A</span></span></span></span></span><span>​</span></span><span><span style="height:0.1774em;"><span></span></span></span></span></span></span><span><span><span><span>SIM</span></span></span></span><span>({</span><span>a</span><span>})</span></span></span></span></span></p>
<p>容差关系，关键是为不完整信息表的所有缺失属性值分配“空”值。“空”值可以是不完整信息表中的任何实际值，丢失的属性值只是“丢失”，但确实存在。</p>
<h2 id="量化容差关系-valued-tolerance-relation"> 量化容差关系（valued tolerance relation）</h2>
<p>Given an IIS <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.10903em;">U</span><span>,</span><span style="margin-right:0.1667em;"></span><span>A</span><span style="margin-right:0.2778em;"></span><span>&gt;</span></span></span></span>, a subset of attributes <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8193em;vertical-align:-0.136em;"></span><span style="margin-right:0.05017em;">B</span><span style="margin-right:0.2778em;"></span><span>⊆</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>. Let <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.07847em;">I</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.10903em;">U</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span>&lt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7335em;vertical-align:-0.1944em;"></span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>x</span><span style="margin-right:0.2778em;"></span><span>&gt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>∣</span><span>x</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10903em;">U</span><span>}</span></span></span></span> be the identity relation on <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">U</span></span></span></span>, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.13889em;">P</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span></span></span></span> be the <strong>tolerance degree</strong>(容差度) between objects <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span> and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span style="margin-right:0.03588em;">y</span></span></span></span> with reference to <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">B</span></span></span></span>, and then for a given threshold <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7335em;vertical-align:-0.0391em;"></span><span>λ</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[</span><span>0</span><span>,</span><span style="margin-right:0.1667em;"></span><span>1</span><span>]</span></span></span></span> the valued tolerance relation <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1244em;vertical-align:-0.2753em;"></span><span style="margin-right:0.22222em;">V</span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.8491em;"><span style="top:-2.4247em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>λ</span></span></span></span></span><span>​</span></span><span><span style="height:0.2753em;"><span></span></span></span></span></span></span></span></span></span> is defined as:</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1461em;vertical-align:-0.247em;"></span><span style="margin-right:0.22222em;">V</span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.8991em;"><span style="top:-2.453em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>λ</span></span></span></span></span><span>​</span></span><span><span style="height:0.247em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span>&lt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7335em;vertical-align:-0.1944em;"></span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span style="margin-right:0.2778em;"></span><span>&gt;∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.1141em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.10903em;">U</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span>∣</span><span><span style="margin-right:0.13889em;">P</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>≥</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>λ</span><span>}</span><span style="margin-right:0.2222em;"></span><span>∪</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.07847em;">I</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.10903em;">U</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>.</span></span></span></span></span></p>
<p>The valued tolerance class <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1244em;vertical-align:-0.2753em;"></span><span style="margin-right:0.22222em;">V</span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.8491em;"><span style="top:-2.4247em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>λ</span></span></span></span></span><span>​</span></span><span><span style="height:0.2753em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span></span></span></span> of object <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span> is defined as:</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1491em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">V</span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.8991em;"><span style="top:-2.453em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>λ</span></span></span></span></span><span>​</span></span><span><span style="height:0.247em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span style="margin-right:0.03588em;">y</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10903em;">U</span><span>∣</span><span style="margin-right:0.2778em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7335em;vertical-align:-0.1944em;"></span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span style="margin-right:0.2778em;"></span><span>&gt;∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.1491em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">V</span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.8991em;"><span style="top:-2.453em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>λ</span></span></span></span></span><span>​</span></span><span><span style="height:0.247em;"><span></span></span></span></span></span></span><span>}</span><span>.</span></span></span></span></span></p>
<p>Obviously, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1244em;vertical-align:-0.2753em;"></span><span style="margin-right:0.22222em;">V</span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.8491em;"><span style="top:-2.4247em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>λ</span></span></span></span></span><span>​</span></span><span><span style="height:0.2753em;"><span></span></span></span></span></span></span></span></span></span> is a <strong>reflexive(自反性) and symmetric(对称性) relation, but not necessarily transitive(传递性)</strong>.</p>
<p>It is noteworthy that we can define different types of <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.13889em;">P</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span></span></span></span> using different comparison rules, and then different types of valued tolerance relations will be got.</p>
<h2 id="二元关系-binary-relation"> 二元关系（binary relation）</h2>
<p>Given sets <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span> and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.22222em;">Y</span></span></span></span>, the binary relation can be defined as an <strong>ordered triple</strong>(有序三元组) <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.00773em;">R</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span><span>,</span><span style="margin-right:0.1667em;"></span><span>G</span><span>(</span><span style="margin-right:0.00773em;">R</span><span>))</span></span></span></span>, where <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>G</span><span>(</span><span style="margin-right:0.00773em;">R</span><span>)</span></span></span></span> is a subset of the <strong>Cartesian product</strong>(笛卡尔积) <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7667em;vertical-align:-0.0833em;"></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.22222em;">Y</span></span></span></span>, which is defined as <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>:</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span>x</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>∧</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.7335em;vertical-align:-0.1944em;"></span><span style="margin-right:0.03588em;">y</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">Y</span><span>}</span></span></span></span>. <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>G</span><span>(</span><span style="margin-right:0.00773em;">R</span><span>)</span></span></span></span> is called the graph(图) of the binary relation. The statement <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>G</span><span>(</span><span style="margin-right:0.00773em;">R</span><span>)</span></span></span></span> is denoted by <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span>x</span><span style="margin-right:0.00773em;">R</span><span style="margin-right:0.03588em;">y</span></span></span></span> or <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.00773em;">R</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span></span></span></span>.</p>
<p><strong>关系矩阵</strong></p>
<p>设<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>}</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>}</span></span></span></span>，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.00773em;">R</span></span></span></span>是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.22222em;">Y</span></span></span></span>上的关系，令：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7167em;vertical-align:-0.2861em;"></span><span><span style="margin-right:0.02778em;">r</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span style="margin-right:0.05724em;">ij</span></span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:3em;vertical-align:-1.25em;"></span><span><span style="top:0em;"><span>{</span></span><span><span><span><span><span><span style="height:1.69em;"><span style="top:-3.69em;"><span style="height:3.008em;"></span><span><span>1</span></span></span><span style="top:-2.25em;"><span style="height:3.008em;"></span><span><span>0</span></span></span></span><span>​</span></span><span><span style="height:1.19em;"><span></span></span></span></span></span><span style="width:1em;"></span><span><span><span><span style="height:1.69em;"><span style="top:-3.69em;"><span style="height:3.008em;"></span><span><span>(</span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span><span style="margin-right:0.00773em;">R</span></span></span><span style="top:-2.25em;"><span style="height:3.008em;"></span><span><span>(</span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span><span><span>∈</span></span><span><span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>/</span><span style="margin-right:0.0556em;"></span></span></span><span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span style="margin-right:0.00773em;">R</span></span></span></span><span>​</span></span><span><span style="height:1.19em;"><span></span></span></span></span></span></span></span><span></span></span></span></span></span></span></p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.0361em;vertical-align:-0.2861em;"></span><span>(</span><span><span style="margin-right:0.02778em;">r</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span style="margin-right:0.05724em;">ij</span></span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:5.46em;vertical-align:-2.48em;"></span><span><span><span><span><span><span style="height:2.95em;"><span style="top:-2.011em;"><span style="height:3.816em;"></span><span><span>⎣</span></span></span><span style="top:-3.158em;"><span style="height:3.816em;"></span><span style="height:1.816em;width:0.6667em;"><svg xmlns="http://www.w3.org/2000/svg" width='0.6667em' height='1.816em' style='width:0.6667em' viewBox='0 0 666.67 1816' preserveAspectRatio='xMinYMin'><path d='M319 0 H403 V1816 H319z M319 0 H403 V1816 H319z'/></svg></span></span><span style="top:-5.6111em;"><span style="height:3.816em;"></span><span><span>⎡</span></span></span></span><span>​</span></span><span><span style="height:2.45em;"><span></span></span></span></span></span></span><span><span><span><span><span><span style="height:2.98em;"><span style="top:-5.8275em;"><span style="height:3.6875em;"></span><span><span><span style="margin-right:0.02778em;">r</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>11</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-4.6275em;"><span style="height:3.6875em;"></span><span><span><span style="margin-right:0.02778em;">r</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>21</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-2.7675em;"><span style="height:3.6875em;"></span><span><span><span>⋮</span><span style="border-right-width:0em;border-top-width:1.5em;bottom:0em;"></span></span></span></span><span style="top:-1.5675em;"><span style="height:3.6875em;"></span><span><span><span style="margin-right:0.02778em;">r</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>n</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:2.48em;"><span></span></span></span></span></span><span style="width:0.5em;"></span><span style="width:0.5em;"></span><span><span><span><span style="height:2.98em;"><span style="top:-5.8275em;"><span style="height:3.6875em;"></span><span><span><span style="margin-right:0.02778em;">r</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>12</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-4.6275em;"><span style="height:3.6875em;"></span><span><span><span style="margin-right:0.02778em;">r</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>22</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-2.7675em;"><span style="height:3.6875em;"></span><span><span><span>⋮</span><span style="border-right-width:0em;border-top-width:1.5em;bottom:0em;"></span></span></span></span><span style="top:-1.5675em;"><span style="height:3.6875em;"></span><span><span><span style="margin-right:0.02778em;">r</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>n</span><span>2</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:2.48em;"><span></span></span></span></span></span><span style="width:0.5em;"></span><span style="width:0.5em;"></span><span><span><span><span style="height:2.98em;"><span style="top:-5.8275em;"><span style="height:3.6875em;"></span><span><span>…</span></span></span><span style="top:-4.6275em;"><span style="height:3.6875em;"></span><span><span>…</span></span></span><span style="top:-2.7675em;"><span style="height:3.6875em;"></span><span><span><span>⋮</span><span style="border-right-width:0em;border-top-width:1.5em;bottom:0em;"></span></span></span></span><span style="top:-1.5675em;"><span style="height:3.6875em;"></span><span><span>⋯</span></span></span></span><span>​</span></span><span><span style="height:2.48em;"><span></span></span></span></span></span><span style="width:0.5em;"></span><span style="width:0.5em;"></span><span><span><span><span style="height:2.98em;"><span style="top:-5.8275em;"><span style="height:3.6875em;"></span><span><span><span style="margin-right:0.02778em;">r</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>1</span><span>m</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-4.6275em;"><span style="height:3.6875em;"></span><span><span><span style="margin-right:0.02778em;">r</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>2</span><span>m</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-2.7675em;"><span style="height:3.6875em;"></span><span><span><span>⋮</span><span style="border-right-width:0em;border-top-width:1.5em;bottom:0em;"></span></span></span></span><span style="top:-1.5675em;"><span style="height:3.6875em;"></span><span><span><span style="margin-right:0.02778em;">r</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>nm</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:2.48em;"><span></span></span></span></span></span></span></span><span><span><span><span><span style="height:2.95em;"><span style="top:-2.011em;"><span style="height:3.816em;"></span><span><span>⎦</span></span></span><span style="top:-3.158em;"><span style="height:3.816em;"></span><span style="height:1.816em;width:0.6667em;"><svg xmlns="http://www.w3.org/2000/svg" width='0.6667em' height='1.816em' style='width:0.6667em' viewBox='0 0 666.67 1816' preserveAspectRatio='xMinYMin'><path d='M263 0 H347 V1816 H263z M263 0 H347 V1816 H263z'/></svg></span></span><span style="top:-5.6111em;"><span style="height:3.816em;"></span><span><span>⎤</span></span></span></span><span>​</span></span><span><span style="height:2.45em;"><span></span></span></span></span></span></span></span></span></span></span></span></p>
<p>该0,1矩阵称为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.00773em;">R</span></span></span></span>的<strong>关系矩阵</strong>，记作<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.10903em;">M</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.109em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.00773em;">R</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>.</p>
<p><strong>关系图</strong></p>
<p>设<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>}</span></span></span></span>，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.00773em;">R</span></span></span></span>是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>上的关系，令图<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>G</span><span>(</span><span style="margin-right:0.00773em;">R</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span style="margin-right:0.22222em;">V</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.05764em;">E</span><span>)</span></span></span></span>，其中顶点集合<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.22222em;">V</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>，边集合为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05764em;">E</span></span></span></span>，且对任意的<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8252em;vertical-align:-0.2861em;"></span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.22222em;">V</span></span></span></span>，满足<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.0361em;vertical-align:-0.2861em;"></span><span>(</span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.05764em;">E</span></span></span></span>当且仅当<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.00773em;">R</span></span></span></span>，称图<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>G</span></span></span></span>是关系<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.00773em;">R</span></span></span></span>的关系图，记作<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span>G</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.00773em;">R</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>。</p>
<p><strong>关系性质</strong></p>
<ul>
<li>
<p>自反性（<strong>reflexive</strong>）：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7335em;vertical-align:-0.0391em;"></span><span>∀</span><span>x</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>A</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.00773em;">R</span></span></span></span></p>
<p>关系图，每个顶点都有环，有从该点自身到自身的有向边。</p>
</li>
<li>
<p>对称性（<strong>symmetric</strong>）：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span>∀</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>A</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7073em;vertical-align:-0.024em;"></span><span style="margin-right:0.00773em;">R</span><span style="margin-right:0.2778em;"></span><span>⟹</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.00773em;">R</span></span></span></span></p>
<p>关系图中，如果有两个顶点之间有边，则一定是一对方向相反的边</p>
</li>
<li>
<p>传递性（<strong>transitive</strong>）：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span>∀</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.04398em;">z</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>A</span><span>,</span><span style="margin-right:0.1667em;"></span><span>((</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.00773em;">R</span><span style="margin-right:0.2222em;"></span><span>∧</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.04398em;">z</span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.00773em;">R</span><span>)</span><span style="margin-right:0.2778em;"></span><span>⟹</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.04398em;">z</span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.00773em;">R</span></span></span></span></p>
<p>关系图中，如果顶点a到b有边，b到c有边，则a到c有边。</p>
</li>
</ul>
<h2 id="数据驱动的量化容差关系-data-driven-valued-tolerance-relation"> 数据驱动的量化容差关系（Data-Driven Valued Tolerance Relation）</h2>
<p><strong>Calculation of tolerance degree</strong>:<sup></sup></p>
<p>Consider an attribute <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span>b</span></span></span></span> of an IIS <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.10903em;">U</span><span>,</span><span style="margin-right:0.1667em;"></span><span>A</span><span style="margin-right:0.2778em;"></span><span>&gt;</span></span></span></span> and the set <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.035em;vertical-align:-0.2831em;"></span><span><span style="margin-right:0.22222em;">V</span><span><span><span><span style="height:0.7519em;"><span style="top:-2.4169em;margin-left:-0.2222em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>′</span></span></span></span></span><span>​</span></span><span><span style="height:0.2831em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.0972em;vertical-align:-0.2831em;"></span><span>{</span><span><span style="margin-right:0.03148em;">k</span><span><span><span><span style="height:0.8141em;"><span style="top:-2.4169em;margin-left:-0.0315em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.2831em;"><span></span></span></span></span></span></span><span><span>b</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03148em;">k</span><span><span><span><span style="height:0.8141em;"><span style="top:-2.4169em;margin-left:-0.0315em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.2831em;"><span></span></span></span></span></span></span><span><span>b</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03148em;">k</span><span><span><span><span style="height:0.6644em;"><span style="top:-2.4169em;margin-left:-0.0315em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:0.2831em;"><span></span></span></span></span></span></span><span><span>b</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>}</span></span></span></span>, where <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span>b</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>b</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>b</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span> are all possible <strong>known values</strong> of <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span>b</span></span></span></span>, and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1078em;vertical-align:-0.2831em;"></span><span><span style="margin-right:0.03148em;">k</span><span><span><span><span style="height:0.8247em;"><span style="top:-2.4169em;margin-left:-0.0315em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.2831em;"><span></span></span></span></span></span></span></span></span></span> denotes the <strong>cardinality</strong>(基数/势，集合中元素个数) of the set <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span>x</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10903em;">U</span><span>∣</span><span>b</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>b</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>}</span></span></span></span>.</p>
<blockquote>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1078em;vertical-align:-0.2831em;"></span><span><span style="margin-right:0.03148em;">k</span><span><span><span><span style="height:0.8247em;"><span style="top:-2.4169em;margin-left:-0.0315em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.2831em;"><span></span></span></span></span></span></span></span></span></span> 是属性值<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8444em;vertical-align:-0.15em;"></span><span><span>b</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>在该属性下的频数</p>
</blockquote>
<p>For any <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span>x</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">U</span></span></span></span>, the probability that <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>b</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8444em;vertical-align:-0.15em;"></span><span><span>b</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span> is <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1078em;vertical-align:-0.2831em;"></span><span><span style="margin-right:0.03148em;">k</span><span><span><span><span style="height:0.8247em;"><span style="top:-2.4169em;margin-left:-0.0315em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.2831em;"><span></span></span></span></span></span></span><span>/</span><span>(</span><span><span style="margin-right:0.03148em;">k</span><span><span><span><span style="height:0.8141em;"><span style="top:-2.4169em;margin-left:-0.0315em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.2831em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.0972em;vertical-align:-0.2831em;"></span><span><span style="margin-right:0.03148em;">k</span><span><span><span><span style="height:0.8141em;"><span style="top:-2.4169em;margin-left:-0.0315em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.2831em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>⋯</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.0331em;vertical-align:-0.2831em;"></span><span><span style="margin-right:0.03148em;">k</span><span><span><span><span style="height:0.6644em;"><span style="top:-2.4169em;margin-left:-0.0315em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:0.2831em;"><span></span></span></span></span></span></span><span>)</span></span></span></span>. Thus, the tolerance degree <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1052em;vertical-align:-0.3552em;"></span><span><span style="margin-right:0.13889em;">P</span><span><span><span><span style="height:0.3448em;"><span style="top:-2.5198em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>{</span><span>b</span><span>}</span></span></span></span></span><span>​</span></span><span><span style="height:0.3552em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span></span></span></span>can be denoted as:</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.1052em;vertical-align:-0.3552em;"></span><span><span style="margin-right:0.13889em;">P</span><span><span><span><span style="height:0.3448em;"><span style="top:-2.5198em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>{</span><span>b</span><span>}</span></span></span></span></span><span>​</span></span><span><span style="height:0.3552em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:7.9706em;vertical-align:-3.7353em;"></span><span><span><span><span><span><span style="height:4.15em;"><span style="top:-1.366em;"><span style="height:4.116em;"></span><span><span>⎩</span></span></span><span style="top:-1.358em;"><span style="height:4.116em;"></span><span style="height:2.116em;width:0.8889em;"><svg xmlns="http://www.w3.org/2000/svg" width='0.8889em' height='2.116em' style='width:0.8889em' viewBox='0 0 888.89 2116' preserveAspectRatio='xMinYMin'><path d='M384 0 H504 V2116 H384z M384 0 H504 V2116 H384z'/></svg></span></span><span style="top:-4.116em;"><span style="height:4.116em;"></span><span><span>⎨</span></span></span><span style="top:-5.258em;"><span style="height:4.116em;"></span><span style="height:2.116em;width:0.8889em;"><svg xmlns="http://www.w3.org/2000/svg" width='0.8889em' height='2.116em' style='width:0.8889em' viewBox='0 0 888.89 2116' preserveAspectRatio='xMinYMin'><path d='M384 0 H504 V2116 H384z M384 0 H504 V2116 H384z'/></svg></span></span><span style="top:-7.366em;"><span style="height:4.116em;"></span><span><span>⎧</span></span></span></span><span>​</span></span><span><span style="height:3.65em;"><span></span></span></span></span></span></span><span><span><span><span><span><span style="height:4.2353em;"><span style="top:-6.739em;"><span style="height:3.5117em;"></span><span><span>1</span></span></span><span style="top:-5.299em;"><span style="height:3.5117em;"></span><span><span>0</span></span></span><span style="top:-3.3553em;"><span style="height:3.5117em;"></span><span><span><span style="margin-right:0.03148em;">k</span><span><span><span><span style="height:0.8247em;"><span style="top:-2.4169em;margin-left:-0.0315em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.2831em;"><span></span></span></span></span></span></span><span>/</span><span style="margin-right:0.1667em;"></span><span><span><span><span style="height:1.5117em;"><span style="top:-2.1223em;margin-left:0em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.05724em;">j</span><span>=</span><span>1</span></span></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>∑</span></span></span><span style="top:-3.95em;margin-left:0em;"><span style="height:3em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:1.1138em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03148em;">k</span><span><span><span><span style="height:0.9426em;"><span style="top:-2.3987em;margin-left:-0.0315em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span><span style="top:-3.1809em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.3013em;"><span></span></span></span></span></span></span></span></span><span style="top:-0.8901em;"><span style="height:3.5117em;"></span><span><span><span><span><span style="height:1.3514em;"><span style="top:-2.1223em;margin-left:0em;"><span style="height:3em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>∑</span></span></span><span style="top:-3.95em;margin-left:0em;"><span style="height:3em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:0.9777em;"><span></span></span></span></span></span><span>(</span><span><span style="margin-right:0.03148em;">k</span><span><span><span><span style="height:0.8247em;"><span style="top:-2.4169em;margin-left:-0.0315em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.2831em;"><span></span></span></span></span></span></span><span>/</span><span style="margin-right:0.1667em;"></span><span><span><span><span style="height:1.3514em;"><span style="top:-2.1223em;margin-left:0em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.05724em;">j</span><span>=</span><span>1</span></span></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>∑</span></span></span><span style="top:-3.95em;margin-left:0em;"><span style="height:3em;"></span><span><span>m</span></span></span></span><span>​</span></span><span><span style="height:1.1138em;"><span></span></span></span></span></span><span><span>)</span><span><span><span><span style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:3.7353em;"><span></span></span></span></span></span><span style="width:1em;"></span><span><span><span><span style="height:4.2353em;"><span style="top:-6.739em;"><span style="height:3.5117em;"></span><span><span>b</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>b</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2222em;"></span><span>∧</span><span style="margin-right:0.2222em;"></span><span>b</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span><span></span></span></span><span></span></span></span></span></span><span>=</span></span><span style="margin-right:0.2778em;"></span><span>∗</span><span style="margin-right:0.2222em;"></span><span>∧</span><span style="margin-right:0.2222em;"></span><span>b</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span><span></span></span></span><span></span></span></span></span></span><span>=</span></span><span style="margin-right:0.2778em;"></span><span>∗</span></span></span><span style="top:-5.299em;"><span style="height:3.5117em;"></span><span><span>b</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span><span></span></span></span><span></span></span></span></span></span><span>=</span></span><span style="margin-right:0.2778em;"></span><span>b</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2222em;"></span><span>∧</span><span style="margin-right:0.2222em;"></span><span>b</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span><span></span></span></span><span></span></span></span></span></span><span>=</span></span><span style="margin-right:0.2778em;"></span><span>∗</span><span style="margin-right:0.2222em;"></span><span>∧</span><span style="margin-right:0.2222em;"></span><span>b</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span><span></span></span></span><span></span></span></span></span></span><span>=</span></span><span style="margin-right:0.2778em;"></span><span>∗</span></span></span><span style="top:-3.3553em;"><span style="height:3.5117em;"></span><span><span>(</span><span>b</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span>b</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>∧</span><span style="margin-right:0.2222em;"></span><span>b</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>∗</span><span>)</span><span style="margin-right:0.2222em;"></span><span>∨</span><span style="margin-right:0.2222em;"></span><span>(</span><span>b</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>∗</span><span style="margin-right:0.2222em;"></span><span>∧</span><span style="margin-right:0.2222em;"></span><span>b</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span>b</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span></span></span><span style="top:-0.8901em;"><span style="height:3.5117em;"></span><span><span>b</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>∗</span><span style="margin-right:0.2222em;"></span><span>∧</span><span style="margin-right:0.2222em;"></span><span>b</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>∗</span></span></span></span><span>​</span></span><span><span style="height:3.7353em;"><span></span></span></span></span></span></span></span><span></span></span></span></span></span></span></p>
<p>In the above formula, the frequency of attribute value is approximately taken as its probability of appearance, and then the approximate probability distribution among the attribute values is got.</p>
<p>This calculation method of tolerance degree does not require any prior knowledge(先验知识) except data sets and is objective(客观的).</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.13889em;">P</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.1052em;vertical-align:-0.3552em;"></span><span><span>Π</span><span><span><span><span style="height:0.3361em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>b</span><span>∈</span><span style="margin-right:0.05017em;">B</span></span></span></span></span><span>​</span></span><span><span style="height:0.1774em;"><span></span></span></span></span></span></span><span><span style="margin-right:0.13889em;">P</span><span><span><span><span style="height:0.3448em;"><span style="top:-2.5198em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>{</span><span>b</span><span>}</span></span></span></span></span><span>​</span></span><span><span style="height:0.3552em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.10903em;">N</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.109em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span></span></span></span></span></p>
<p>where <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.10903em;">N</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.109em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span></span></span></span> is the weight factor of objects <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span> and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span style="margin-right:0.03588em;">y</span></span></span></span> taking the known and same attribute values on <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">B</span></span></span></span>, i.e.,</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.10903em;">N</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.109em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.363em;vertical-align:-0.936em;"></span><span><span></span><span><span><span><span style="height:1.427em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>∣</span><span style="margin-right:0.05017em;">B</span><span>∣</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>∣</span><span>{</span><span>b</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span><span style="margin-right:0.05017em;">B</span><span>∣</span><span>b</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>b</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span><span></span></span></span><span></span></span></span></span></span><span>=</span></span><span style="margin-right:0.2778em;"></span><span>∗</span><span>}</span><span>∣</span></span></span></span><span>​</span></span><span><span style="height:0.936em;"><span></span></span></span></span></span><span></span></span></span></span></span></span></p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>∣</span><span style="margin-right:0.05017em;">B</span><span>∣</span></span></span></span>表示该集合中不重复元素的个数。</p>
<p><strong>Example 1.</strong> Suppose an IIS <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.10903em;">U</span><span>,</span><span style="margin-right:0.1667em;"></span><span>A</span><span style="margin-right:0.2778em;"></span><span>&gt;</span></span></span></span> is given in the below table, where <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>12</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span> are the available objects,<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>3</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>4</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span> are four attributes.</p>
<center><img src=".\images\table-1.png" width=80%></center>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.13889em;">P</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.0074em;vertical-align:-0.686em;"></span><span>(</span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>10</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>4</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:2.0074em;vertical-align:-0.686em;"></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>6</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>3</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:2.0074em;vertical-align:-0.686em;"></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>8</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>4</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>1</span><span>)</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:2.0074em;vertical-align:-0.686em;"></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>4</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.0074em;vertical-align:-0.686em;"></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>40</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span></span></span></span></span></p>
<div><p>提示</p>
<p>计算结果说明</p>
<p>其中，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>4/10</span></span></span></span>是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>属性上，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>之间已知的属性出现的值(1) 在整个<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>已知属性值集合元素个数(10)的频数。<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>3/6</span></span></span></span>是在<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>属性上，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>之间已知的属性出现的值(2)在整个<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>属性值集合元素个数(6)的频数，其它同理。</p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.10903em;">N</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.109em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.1901em;vertical-align:-0.345em;"></span><span><span></span><span><span><span><span style="height:0.8451em;"><span style="top:-2.655em;"><span style="height:3em;"></span><span><span><span>4</span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.394em;"><span style="height:3em;"></span><span><span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.345em;"><span></span></span></span></span></span><span></span></span></span></span></span>表示在属性集<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>3</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>c</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>4</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>}</span></span></span></span>上，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7335em;vertical-align:-0.1944em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>&gt;</span></span></span></span>都是已知属性值对的个数(1)。</p>
</div>
<p>and</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.13889em;">P</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>3</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>x</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>4</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.0074em;vertical-align:-0.686em;"></span><span>(</span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>10</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.7278em;vertical-align:-0.0833em;"></span><span>1</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.7278em;vertical-align:-0.0833em;"></span><span>1</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>1</span><span>)</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:2.0074em;vertical-align:-0.686em;"></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>4</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>3</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.0074em;vertical-align:-0.686em;"></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>40</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>3</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span></span></span></span></span></p>
<p><strong>Auto-selection of Threshold:</strong></p>
<p>For any <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span>x</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span style="margin-right:0.10903em;">U</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.05017em;">B</span><span style="margin-right:0.2778em;"></span><span>⊆</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>, the tolerance class <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span></span></span></span> can be calculated. For any <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7335em;vertical-align:-0.1944em;"></span><span style="margin-right:0.03588em;">y</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span></span></span></span>, the tolerance degree <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.13889em;">P</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span></span></span></span> can be calculated.</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:3em;vertical-align:-1.25em;"></span><span><span><span><span><span style="height:1.75em;"><span style="top:-3.91em;"><span style="height:3em;"></span><span><span><span>λ</span><span><span><span><span style="height:0.7144em;"><span style="top:-2.453em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span><span>max</span></span></span></span></span><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>x</span></span></span></span><span>​</span></span><span><span style="height:0.247em;"><span></span></span></span></span></span></span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span><span>λ</span><span><span><span><span style="height:0.7144em;"><span style="top:-2.453em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span><span>min</span></span></span></span></span><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>x</span></span></span></span><span>​</span></span><span><span style="height:0.247em;"><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:1.25em;"><span></span></span></span></span></span><span><span><span><span style="height:1.75em;"><span style="top:-3.91em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span>max</span></span></span><span>{</span><span><span style="margin-right:0.13889em;">P</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span>∣</span><span style="margin-right:0.03588em;">y</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)}</span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span>min</span></span></span><span>{</span><span><span style="margin-right:0.13889em;">P</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span>∣</span><span style="margin-right:0.03588em;">y</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05017em;">B</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)}</span></span></span></span><span>​</span></span><span><span style="height:1.25em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:1.75em;"><span style="top:-3.75em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-2.25em;"><span style="height:2.84em;"></span><span></span></span></span><span>​</span></span><span><span style="height:1.25em;"><span></span></span></span></span></span></span></span></span></p>
<p>Thus we may assume that:</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:3em;vertical-align:-1.25em;"></span><span><span><span><span><span style="height:1.75em;"><span style="top:-3.91em;"><span style="height:3em;"></span><span><span><span>λ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span><span>max</span></span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span><span>λ</span><span><span><span><span style="height:0.3175em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span><span>min</span></span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:1.25em;"><span></span></span></span></span></span><span><span><span><span style="height:1.75em;"><span style="top:-3.91em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span>min</span></span></span><span>{</span><span><span>λ</span><span><span><span><span style="height:0.7144em;"><span style="top:-2.453em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span><span>max</span></span></span></span></span><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>x</span></span></span></span><span>​</span></span><span><span style="height:0.247em;"><span></span></span></span></span></span></span><span>∣</span><span>x</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span><span style="margin-right:0.10903em;">U</span><span>}</span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span><span>max</span></span></span><span>{</span><span><span>λ</span><span><span><span><span style="height:0.7144em;"><span style="top:-2.453em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span><span>min</span></span></span></span></span><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>x</span></span></span></span><span>​</span></span><span><span style="height:0.247em;"><span></span></span></span></span></span></span><span>∣</span><span>x</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span><span style="margin-right:0.10903em;">U</span><span>}</span></span></span></span><span>​</span></span><span><span style="height:1.25em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:1.75em;"><span style="top:-3.75em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-2.25em;"><span style="height:2.84em;"></span><span></span></span></span><span>​</span></span><span><span style="height:1.25em;"><span></span></span></span></span></span></span></span></span></p>
<p>Let:</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span>λ</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>min</span></span></span><span>{</span><span><span>λ</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span><span>max</span></span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span>λ</span><span><span><span><span style="height:0.3175em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span><span>min</span></span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>}</span></span></span></span></span></p>
<h2 id="特征关系-characteristic-relation"> 特征关系（characteristic relation）</h2>
<p>We will consider four kinds of missing attribute values, called <strong>lost</strong>(丢失型), <strong>&quot;do not care&quot; conditions</strong>(不关系条件型), <strong>restricted &quot;do not care&quot; conditions</strong>(限制不关心条件型) and <strong>attribute-concept values</strong>(属性-概念值)<sup></sup>.</p>
<p>We will assume that lost values will be denoted by &quot;?&quot;, &quot;do not care&quot; conditions by &quot;<code>*</code>&quot;, restricted &quot;do not care&quot; conditions by &quot;+&quot;, and attribute-concept values by &quot;-&quot;.</p>
<p>An important tool to analyze complete decision tables is <strong>a block of the attribute-value pair</strong>. Let <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span> be an attribute, and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span>a</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>. Let <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.03588em;">v</span></span></span></span> be a value of <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span> for some cases.</p>
<p>For complete decision tables if <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6151em;"></span><span>t</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span>a</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">v</span><span>)</span></span></span></span> is an attribute-value pair, then a block of <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6151em;"></span><span>t</span></span></span></span>, denoted <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[</span><span>t</span><span>]</span></span></span></span>, is a set of all cases from <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">U</span></span></span></span> that for attribute <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span> has value <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.03588em;">v</span></span></span></span>.</p>
<blockquote>
<p>对不完备信息系统而言，只有已知属性值才有属性值对块，未知属性值没有属性值对块。</p>
</blockquote>
<p>Additionally, any decision table defines a <strong>function</strong> <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>ρ</span></span></span></span> that maps the direct product(乘积) of <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">U</span></span></span></span> and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span></span></span></span> into the set of all values.</p>
<ul>
<li>
<p>If for an attribute <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span> there exists a case <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span> such that <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span></span><span><span style="height:0.6944em;"></span><span>?</span></span></span></span>, i.e., the corresponding value is lost, then the case <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span> should not be included in any blocks <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span>a</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">v</span><span>)]</span></span></span></span> for all values <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.03588em;">v</span></span></span></span> of attribute <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span>.</p>
<p>丢失型，该case不包括在任何的属性值对块中的。</p>
</li>
<li>
<p>If for an attribute <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span> there exists a case <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span> such that the corresponding value is a &quot;do not care&quot; condition or a restricted &quot;do not care&quot; condition, i.e., <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.4653em;"></span><span>∗</span></span></span></span> or <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>+</span></span></span></span>, then the case <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span> should be included in blocks <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span>a</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">v</span><span>)]</span></span></span></span> for all specified values <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.03588em;">v</span></span></span></span> of attribute <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span>.</p>
<p>不关系条件型或者限制不关系条件型，该case在属性<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span>上的值，可以是该<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span>属性值集中的任一个。</p>
</li>
<li>
<p>If for an attribute <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span> there exists a case <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span> such that the corresponding value is an attribute-concept value, i.e., <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>−</span></span></span></span>, then the corresponding case <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span> should be included in blocks <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span>a</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">v</span><span>)]</span></span></span></span> for all specified values <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span style="margin-right:0.03588em;">v</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">V</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span></span></span></span> of attribute <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span>, where:</p>
</li>
</ul>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">V</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span>ρ</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span><span> </span><span>∣</span><span>ρ</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span><span>i</span><span>s</span><span> </span><span>s</span><span>p</span><span>ec</span><span>i</span><span style="margin-right:0.10764em;">f</span><span>i</span><span>e</span><span>d</span><span>,</span><span> </span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10903em;">U</span><span>,</span><span style="margin-right:0.1667em;"></span><span>ρ</span><span>(</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span>d</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>d</span><span>)}</span><span>.</span></span></span></span></span></p>
<p>​        即当case <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span>和case <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span style="margin-right:0.03588em;">y</span></span></span></span> 都有相同的决策属性值<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span>d</span></span></span></span>， 且满足<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>−</span></span></span></span>，即是属性-概念值，那么case <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span>在<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span>属性上的属性值是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span>?</span></span></span></span>，该值可以用case <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span style="margin-right:0.03588em;">y</span></span></span></span> 在<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span>属性上的确定的属性值来代替。</p>
<p><strong>特征集（characteristic set）</strong></p>
<p>For a case <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span>x</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">U</span></span></span></span>, the characteristic set <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span></span></span></span> is defined as the intersection(交集) of the sets <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.07153em;">K</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span></span></span></span>, for all <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span>a</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>, where the set <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.07153em;">K</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span></span></span></span> is defined in the following way:</p>
<ul>
<li>
<p>If <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span></span></span></span> is specified, then <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.07153em;">K</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span></span></span></span> is the block <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span>a</span><span>,</span><span style="margin-right:0.1667em;"></span><span>ρ</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>))]</span></span></span></span> of attribute <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span> and its value <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span></span></span></span>.</p>
<p>属性值确定，则是属性值对。</p>
</li>
<li>
<p>If <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span></span><span><span style="height:0.6944em;"></span><span>?</span></span></span></span> or <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.4653em;"></span><span>∗</span></span></span></span> then the set <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.07153em;">K</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">U</span></span></span></span>.</p>
<p>是丢失型或者不关心条件型，则是整个论域</p>
</li>
<li>
<p>If $\rho(x,a) = + $, then <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.07153em;">K</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span></span></span></span> is equal to the <strong>union</strong>(并集) of all blocks of <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span>a</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">v</span><span>)</span></span></span></span>, for all specified values <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.03588em;">v</span></span></span></span> of attribute <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>a</span></span></span></span>.</p>
<p>是限制不关系条件型，是所有<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span>a</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">v</span><span>)</span></span></span></span>块的的并集</p>
</li>
<li>
<p>If <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>−</span></span></span></span>, then the corresponding set <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.07153em;">K</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span></span></span></span> is equal to the union of all blocks of attribute-value pair <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span>a</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">v</span><span>)</span></span></span></span>, where <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5782em;vertical-align:-0.0391em;"></span><span style="margin-right:0.03588em;">v</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">V</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span></span></span></span> if <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">V</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span></span></span></span> is nonempty. If <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">V</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span></span></span></span> is empty, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.07153em;">K</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">U</span></span></span></span>.</p>
<p>是属性-值对型，空集情况下是整个论域；非空情况下，是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span>a</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">v</span><span>)</span></span></span></span>块的并集。</p>
</li>
</ul>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span></span></span></span> may be interpreted as the set of cases that ate indistinguishable from <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span> using all attributes from <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span></span></span></span> and using a given interpretation of missing attribute values.</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>∩</span><span style="margin-right:0.07153em;">K</span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span>a</span><span>)</span></span></span></span></span></p>
<blockquote>
<p>特征集<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span></span></span></span>可以理解为与实例<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span>不可分辨的实例的集合，这些与实例<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span>不可分辨的实例是利用实例<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span>中所有属于属性集<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>的属性。</p>
</blockquote>
<p><strong>特征关系（characteristic relation）</strong></p>
<p>The characteristic relation <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.00773em;">R</span><span>(</span><span>A</span><span>)</span></span></span></span> is a relation on <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">U</span></span></span></span> defined for <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7335em;vertical-align:-0.1944em;"></span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6833em;"></span><span style="margin-right:0.10903em;">U</span></span></span></span> as follows<sup></sup>:</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>(</span><span>x</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.00773em;">R</span><span>(</span><span>A</span><span>)</span><span style="margin-right:1em;"></span><span>i</span><span style="margin-right:0.10764em;">f</span><span> </span><span>an</span><span>d</span><span> </span><span>o</span><span>n</span><span style="margin-right:0.01968em;">l</span><span style="margin-right:0.03588em;">y</span><span> </span><span>i</span><span style="margin-right:0.10764em;">f</span><span style="margin-right:1em;"></span><span style="margin-right:0.03588em;">y</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span></span></span></span></span></p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.00773em;">R</span><span>(</span><span>A</span><span>)</span></span></span></span> is reflexive but, in general, does not need to be symmetric or transitive.(是自反性，不满足对称性和传递性)</p>
<p><strong>Example 2.</strong></p>
<center><img src=".\images\table-2.png" width=80%></center>
<p>For case 1, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10903em;">N</span><span>o</span><span>i</span><span>se</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>−</span></span></span></span>, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>Q</span><span>u</span><span>a</span><span style="margin-right:0.01968em;">l</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span>hi</span><span style="margin-right:0.03588em;">g</span><span>h</span></span></span></span>, and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>7</span><span>,</span><span style="margin-right:0.1667em;"></span><span>Q</span><span>u</span><span>a</span><span style="margin-right:0.01968em;">l</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span>hi</span><span style="margin-right:0.03588em;">g</span><span>h</span></span></span></span>, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>7</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10903em;">N</span><span>o</span><span>i</span><span>se</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6944em;"></span><span style="margin-right:0.01968em;">l</span><span>o</span><span style="margin-right:0.02691em;">w</span></span></span></span>. So, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">V</span><span>(</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10903em;">N</span><span>o</span><span>i</span><span>se</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span> </span><span style="margin-right:0.01968em;">l</span><span>o</span><span style="margin-right:0.02691em;">w</span><span>}</span></span></span></span>, we add case 1 to <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.10903em;">N</span><span>o</span><span>i</span><span>se</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.01968em;">l</span><span>o</span><span style="margin-right:0.02691em;">w</span><span>)]</span></span></span></span>.【根据相同decision属性值来确定attribute-concept values】</p>
<p>For case 2, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10903em;">N</span><span>o</span><span>i</span><span>se</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.4653em;"></span><span>∗</span></span></span></span>, hence case 2 is included in both sets: <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.10903em;">N</span><span>o</span><span>i</span><span>se</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.01968em;">l</span><span>o</span><span style="margin-right:0.02691em;">w</span><span>)]</span></span></span></span> and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.10903em;">N</span><span>o</span><span>i</span><span>se</span><span>,</span><span style="margin-right:0.1667em;"></span><span>m</span><span>e</span><span>d</span><span>i</span><span>u</span><span>m</span><span>)]</span></span></span></span>.【不关心条件型，可以是属性值对中的任一个】</p>
<p>Similarly, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.05764em;">S</span><span>i</span><span>ze</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.4653em;"></span><span>∗</span></span></span></span>, hence case 2 is included in all three sets: <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.05764em;">S</span><span>i</span><span>ze</span><span>,</span><span style="margin-right:0.1667em;"></span><span>co</span><span>m</span><span>p</span><span>a</span><span>c</span><span>t</span><span>)]</span></span></span></span>, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.05764em;">S</span><span>i</span><span>ze</span><span>,</span><span style="margin-right:0.1667em;"></span><span>m</span><span>e</span><span>d</span><span>i</span><span>u</span><span>m</span><span>)]</span></span></span></span> and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.05764em;">S</span><span>i</span><span>ze</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10764em;">f</span><span>u</span><span style="margin-right:0.01968em;">ll</span><span>)]</span></span></span></span>.</p>
<p>Furthermore, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span></span><span><span style="height:0.6944em;"></span><span>?</span></span></span></span>, so case 3 is not included in <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02691em;">tw</span><span>o</span><span>)]</span></span></span></span>,  <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10764em;">f</span><span>o</span><span>u</span><span style="margin-right:0.02778em;">r</span><span>)]</span></span></span></span> and  <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10764em;">f</span><span>i</span><span style="margin-right:0.03588em;">v</span><span>e</span><span>)]</span></span></span></span>.</p>
<p>For case 4, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>+</span></span></span></span>, so case 4 is included in  <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02691em;">tw</span><span>o</span><span>)]</span></span></span></span>,  <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10764em;">f</span><span>o</span><span>u</span><span style="margin-right:0.02778em;">r</span><span>)]</span></span></span></span> and  <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10764em;">f</span><span>i</span><span style="margin-right:0.03588em;">v</span><span>e</span><span>)]</span></span></span></span>.</p>
<p>Also, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>5</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10903em;">N</span><span>o</span><span>i</span><span>se</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span></span><span><span style="height:0.6944em;"></span><span>?</span></span></span></span>, so case 5 is not in <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.10903em;">N</span><span>o</span><span>i</span><span>se</span><span>,</span><span style="margin-right:0.1667em;"></span><span>m</span><span>e</span><span>d</span><span>i</span><span>u</span><span>m</span><span>)]</span></span></span></span>, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.10903em;">N</span><span>o</span><span>i</span><span>se</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.01968em;">l</span><span>o</span><span style="margin-right:0.02691em;">w</span><span>)]</span></span></span></span>.</p>
<p>Finally, <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>ρ</span><span>(</span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>−</span></span></span></span>, and <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.22222em;">V</span><span>(</span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8056em;vertical-align:-0.0556em;"></span><span>∅</span></span></span></span>(没有和case 6决策属性值一样的在Capacity属性上属性值明确的case), so case 6 is not included in  <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02691em;">tw</span><span>o</span><span>)]</span></span></span></span>,  <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10764em;">f</span><span>o</span><span>u</span><span style="margin-right:0.02778em;">r</span><span>)]</span></span></span></span> and  <span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10764em;">f</span><span>i</span><span style="margin-right:0.03588em;">v</span><span>e</span><span>)]</span></span></span></span>.</p>
<p>Thus, case =<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>5</span><span>,</span><span style="margin-right:0.1667em;"></span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span>7</span><span>}</span></span></span></span></p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:12em;vertical-align:-5.75em;"></span><span><span><span><span><span style="height:6.25em;"><span style="top:-8.41em;"><span style="height:3em;"></span><span><span>[(</span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02691em;">tw</span><span>o</span><span>)]</span></span></span><span style="top:-6.91em;"><span style="height:3em;"></span><span><span>[(</span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10764em;">f</span><span>o</span><span>u</span><span style="margin-right:0.02778em;">r</span><span>)]</span></span></span><span style="top:-5.41em;"><span style="height:3em;"></span><span><span>[(</span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10764em;">f</span><span>i</span><span style="margin-right:0.03588em;">v</span><span>e</span><span>)]</span></span></span><span style="top:-3.91em;"><span style="height:3em;"></span><span><span>[(</span><span style="margin-right:0.10903em;">N</span><span>o</span><span>i</span><span>se</span><span>,</span><span style="margin-right:0.1667em;"></span><span>m</span><span>e</span><span>d</span><span>i</span><span>u</span><span>m</span><span>)]</span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span>[(</span><span style="margin-right:0.10903em;">N</span><span>o</span><span>i</span><span>se</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.01968em;">l</span><span>o</span><span style="margin-right:0.02691em;">w</span><span>)]</span></span></span><span style="top:-0.91em;"><span style="height:3em;"></span><span><span>[(</span><span style="margin-right:0.05764em;">S</span><span>i</span><span>ze</span><span>,</span><span style="margin-right:0.1667em;"></span><span>co</span><span>m</span><span>p</span><span>a</span><span>c</span><span>t</span><span>)]</span></span></span><span style="top:0.59em;"><span style="height:3em;"></span><span><span>[(</span><span style="margin-right:0.05764em;">S</span><span>i</span><span>ze</span><span>,</span><span style="margin-right:0.1667em;"></span><span>m</span><span>e</span><span>d</span><span>i</span><span>u</span><span>m</span><span>)]</span></span></span><span style="top:2.09em;"><span style="height:3em;"></span><span><span>[(</span><span style="margin-right:0.05764em;">S</span><span>i</span><span>ze</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.10764em;">f</span><span>u</span><span style="margin-right:0.01968em;">ll</span><span>)]</span></span></span></span><span>​</span></span><span><span style="height:5.75em;"><span></span></span></span></span></span><span><span><span><span style="height:6.25em;"><span style="top:-8.41em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>}</span></span></span><span style="top:-6.91em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>5</span><span>}</span></span></span><span style="top:-5.41em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>7</span><span>}</span></span></span><span style="top:-3.91em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span>6</span><span>}</span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>7</span><span>}</span></span></span><span style="top:-0.91em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>}</span></span></span><span style="top:0.59em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span>5</span><span>}</span></span></span><span style="top:2.09em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span>7</span><span>}</span></span></span></span><span>​</span></span><span><span style="height:5.75em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:6.25em;"><span style="top:-8.25em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-6.75em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-5.25em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-3.75em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-2.25em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-0.75em;"><span style="height:2.84em;"></span><span></span></span><span style="top:0.75em;"><span style="height:2.84em;"></span><span></span></span><span style="top:2.25em;"><span style="height:2.84em;"></span><span></span></span></span><span>​</span></span><span><span style="height:5.75em;"><span></span></span></span></span></span></span></span></span></p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>1</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.07153em;">C</span><span>a</span><span>p</span><span>a</span><span>c</span><span>i</span><span>t</span><span style="margin-right:0.03588em;">y</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.02691em;">tw</span><span>o</span><span>)]</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.10903em;">N</span><span>o</span><span>i</span><span>se</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.01968em;">l</span><span>o</span><span style="margin-right:0.02691em;">w</span><span>)]</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>[(</span><span style="margin-right:0.05764em;">S</span><span>i</span><span>ze</span><span>,</span><span style="margin-right:0.1667em;"></span><span>co</span><span>m</span><span>p</span><span>a</span><span>c</span><span>t</span><span>)]</span></span><span></span><span><span style="height:0.3669em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>}</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>7</span><span>}</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>}</span></span><span></span><span><span style="height:0.3669em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>}</span></span></span></span></span></p>
<p>Similarly,</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:9em;vertical-align:-4.25em;"></span><span><span><span><span><span style="height:4.75em;"><span style="top:-6.91em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>2</span><span>)</span></span></span><span style="top:-5.41em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>3</span><span>)</span></span></span><span style="top:-3.91em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>4</span><span>)</span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>5</span><span>)</span></span></span><span style="top:-0.91em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>6</span><span>)</span></span></span><span style="top:0.59em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>7</span><span>)</span></span></span></span><span>​</span></span><span><span style="height:4.25em;"><span></span></span></span></span></span><span><span><span><span style="height:4.75em;"><span style="top:-6.91em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>5</span><span>}</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span><span style="margin-right:0.10903em;">U</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span><span style="margin-right:0.10903em;">U</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>5</span><span>}</span></span></span><span style="top:-5.41em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span style="margin-right:0.10903em;">U</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span>6</span><span>}</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span>5</span><span>}</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>3</span><span>}</span></span></span><span style="top:-3.91em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>({</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>}</span><span style="margin-right:0.2222em;"></span><span>∪</span><span style="margin-right:0.2222em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>5</span><span>}</span><span style="margin-right:0.2222em;"></span><span>∪</span><span style="margin-right:0.2222em;"></span><span>{</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>7</span><span>})</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>7</span><span>}</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>}</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>}</span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>5</span><span>}</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span><span style="margin-right:0.10903em;">U</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span>5</span><span>}</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>5</span><span>}</span></span></span><span style="top:-0.91em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span style="margin-right:0.10903em;">U</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span>6</span><span>}</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span>7</span><span>}</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>6</span><span>}</span></span></span><span style="top:0.59em;"><span style="height:3em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>7</span><span>}</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>7</span><span>}</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span><span>{</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span>7</span><span>}</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>7</span><span>}</span></span></span></span><span>​</span></span><span><span style="height:4.25em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:4.75em;"><span style="top:-6.75em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-5.25em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-3.75em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-2.25em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-0.75em;"><span style="height:2.84em;"></span><span></span></span><span style="top:0.75em;"><span style="height:2.84em;"></span><span></span></span></span><span>​</span></span><span><span style="height:4.25em;"><span></span></span></span></span></span></span></span></span></p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:3em;vertical-align:-1.25em;"></span><span><span><span><span><span style="height:1.75em;"><span style="top:-3.91em;"><span style="height:3em;"></span><span><span>(</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.03588em;">y</span><span>)</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span><span style="margin-right:0.00773em;">R</span><span>(</span><span>A</span><span>)</span><span style="margin-right:1em;"></span><span>i</span><span style="margin-right:0.10764em;">f</span><span> </span><span>an</span><span>d</span><span> </span><span>o</span><span>n</span><span style="margin-right:0.01968em;">l</span><span style="margin-right:0.03588em;">y</span><span> </span><span>i</span><span style="margin-right:0.10764em;">f</span><span style="margin-right:1em;"></span><span style="margin-right:0.03588em;">y</span><span style="margin-right:0.2778em;"></span><span>∈</span><span style="margin-right:0.2778em;"></span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>1</span><span>)</span></span></span><span style="top:-2.41em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.07153em;">K</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>A</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>1</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>{</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>}</span></span></span></span><span>​</span></span><span><span style="height:1.25em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:1.75em;"><span style="top:-3.75em;"><span style="height:2.84em;"></span><span></span></span><span style="top:-2.25em;"><span style="height:2.84em;"></span><span></span></span></span><span>​</span></span><span><span style="height:1.25em;"><span></span></span></span></span></span></span></span></span></p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.00773em;">R</span><span>(</span><span>A</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>{(</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>1</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>5</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>3</span><span>,</span><span style="margin-right:0.1667em;"></span><span>3</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>1</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>4</span><span>,</span><span style="margin-right:0.1667em;"></span><span>4</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>5</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>5</span><span>,</span><span style="margin-right:0.1667em;"></span><span>5</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>6</span><span>,</span><span style="margin-right:0.1667em;"></span><span>6</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span>(</span><span>7</span><span>,</span><span style="margin-right:0.1667em;"></span><span>7</span><span>)}</span></span></span></span></span></p>
<h2 id="references"> References</h2>
<hr>
<section>
<ol>
<li id="footnote1"><p>Pawlak, Z.: Rough sets. International Journal of Computer and Information Sciences 11, 341–356 (1982) </p>
</li>
<li id="footnote2"><p>https://www.geeksforgeeks.org/rough-set-theory-an-introduction/ </p>
</li>
<li id="footnote3"><p>Wang, G., Guan, L. (2012). Data-Driven Valued Tolerance Relation. In: , <em>et al.</em> Rough Sets and Knowledge Technology. RSKT 2012. Lecture Notes in Computer Science(), vol 7414. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31900-6_2   </p>
</li>
<li id="footnote4"><p>M. Kryszkiewicz. Rough set approach to incomplete information systems[J] Information Sciences. 1998, 112(1): 39-49.  </p>
</li>
<li id="footnote5"><p>Slowinski, R., Vanderpooten, D.: A generalized definition of rough approximations based on similarity. IEEE Transactions on Knowledge and Data Engineering 12(2),331–336 (2000) </p>
</li>
<li id="footnote6"><p>Wang, G.Y.: Extension of rough set under incomplete information systems. Journal of Computer Research and Development 39(10), 1238–1243 (2002) </p>
</li>
<li id="footnote7"><p>Stefanowski, J., Tsouki` as, A.: On the Extension of Rough Sets under Incomplete Information. In: Zhong, N., Skowron, A., Ohsuga, S. (eds.) RSFDGrC 1999. LNCS (LNAI), vol. 1711, pp. 73–82. Springer, Heidelberg (1999) </p>
</li>
<li id="footnote8"><p>Grzymala-Busse, J.W.: Characteristic Relations for Incomplete Data: A Generalization of the Indiscernibility Relation. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets IV. LNCS, vol. 3700, pp. 58–68. Springer, Heidelberg (2005)  </p>
</li>
<li id="footnote9"><p>J. W. Grzymala-Busse. Rough set strategies to data with missing attribute values[C]. Proceedings of the 2nd Workshop on Foundations and New Directions of Data Mining, Melbourne, 2003: 197-212. </p>
</li>
</ol>
</section>
]]></content:encoded>
    </item>
    <item>
      <title>先进核能概论</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/nst/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/nst/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">先进核能概论</source>
      <description>整理的关于第四代先进核能的基础知识。</description>
      <content:encoded><![CDATA[<p>主要是面向熔盐堆。</p>
<p>待补充。。。</p>
]]></content:encoded>
    </item>
    <item>
      <title>第四代先进核能系统概述</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/nst/%E5%9B%9B%E4%BB%A3%E5%A0%86%E6%A6%82%E8%BF%B0/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/nst/%E5%9B%9B%E4%BB%A3%E5%A0%86%E6%A6%82%E8%BF%B0/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">第四代先进核能系统概述</source>
      <description>对第四代先进核能系统发展的一个总体概述（简要整理）。</description>
      <category>NuclearScience</category>
      <pubDate>Tue, 01 Mar 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="核能技术发展现状"> 核能技术发展现状</h2>
<p>1、压水堆是核电开发的首选</p>
<p>2、现役机组性能不断改善</p>
<p>3、高龄机组延寿成为趋势</p>
<p>4、三代堆成为核电建设的主流技术</p>
<p>5、小型模块化反应堆（SMR）研发掀起热潮</p>
<p>6、乏燃料管理压力增大，核燃料循环后端需求迫切</p>
<p>7、四代先进核能系统研发情况</p>
<center><img src="./images/四代堆发展现状.jpg" width=80% title="四代堆发展现状">
</center>
<h2 id="核能发展方向"> 核能发展方向</h2>
<p>（一）热堆规模化发展：</p>
<p>1、铀资源勘查程度低。从地址勘查获天然铀，再经铀转化、铀同位素分离和制造核燃料元件制造入堆，该过程需要15年以上。</p>
<p>2、三代先进压水堆安全性和经济性优化平衡。</p>
<p>3、核电设备可靠性、运维技术、老化管理及应急响应技术完善和提高，实现“低端手工式”到“高端智能”转变。</p>
<p>4、核电软件自主化开发能力突破。美和欧盟正在开发“数值反应堆技术”。</p>
<p>5、乏燃料后处理能力提高</p>
<p>（二）快堆和四代堆</p>
<p>1、裂变燃料的增殖，核电长期规模化发展面临燃料供应不足的风险。</p>
<p>2、超铀元素分离和嬗变。超铀元素含有核燃料，是乏燃料长期放射性来源。</p>
<p>3、先进核能多用途利用。发电、供热、海水淡化、核动力领域等。</p>
<p>4、四代堆的优势、技术成熟度和发展空间明确。</p>
<p>（三）聚变堆</p>
<h2 id="核能技术发展路线"> 核能技术发展路线</h2>
<center><img src="./images/核能发展路线.jpg" title="核能发展路线">
</center>
参考资料[1]
<hr>
<h2 id="发展四代堆的必要性"> 发展四代堆的必要性</h2>
<p>1、更好的经济性</p>
<p>三代堆输出约300℃的热，热电转换效率约为33%</p>
<p>四代输出≥500℃的热，热电转换效率（40%~60%）</p>
<p>可综合利用，包括制氢、海水净化、供热、干冷（可建在无水区）</p>
<p>2、更好的安全性</p>
<p>三代堆依靠外部动力去除反应堆停堆后的剩余热</p>
<p>四代堆使用自然循环去除剩余热，追求本征安全性，具有防备恐怖袭击和自然灾害能力</p>
<p>3、提高燃料利用率</p>
<p>三代堆的燃料利用率约为1~2%</p>
<p>四代堆的燃料利用率大于10%，追求完全的燃料循环</p>
<h2 id="四代核能简介"> 四代核能简介</h2>
<center><img src="./images/四代堆划分.jpg" width=80%></center>
<p>三代核电技术主要不足是 核电站建设运营成本高、经济性差，且未有考虑有效防止核扩散。第四代核能系统自2002年第五次第四代核能系统国际论坛（GIF）提出，旨在进一步提高安全性，降低建设及运营成本，考虑防止核扩散的要求，使未来核电更加安全、廉价。</p>
<p>第四代核能系统概念不仅用于发电，还包括制氢及海水淡化等功能。具体堆型特点比较如下[2]：</p>
<table>
<thead>
<tr>
<th style="text-align:center">反应堆</th>
<th style="text-align:center">中子谱</th>
<th style="text-align:center">冷却剂</th>
<th style="text-align:center">出口温度/℃</th>
<th style="text-align:center">燃料循环方式</th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align:center">超临界水冷堆（SCWR）</td>
<td style="text-align:center">热/快</td>
<td style="text-align:center">水</td>
<td style="text-align:center">510-625</td>
<td style="text-align:center">一次通过/闭式</td>
</tr>
<tr>
<td style="text-align:center">超高温气冷堆（VHTR）</td>
<td style="text-align:center">热</td>
<td style="text-align:center">氦</td>
<td style="text-align:center">900-1000</td>
<td style="text-align:center">一次通过</td>
</tr>
<tr>
<td style="text-align:center">熔盐反应堆（MSR）</td>
<td style="text-align:center">热/快</td>
<td style="text-align:center">氟化物盐</td>
<td style="text-align:center">700-800</td>
<td style="text-align:center">闭式</td>
</tr>
<tr>
<td style="text-align:center">钠冷快堆（SFR）</td>
<td style="text-align:center">快</td>
<td style="text-align:center">钠</td>
<td style="text-align:center">550</td>
<td style="text-align:center">闭式</td>
</tr>
<tr>
<td style="text-align:center">铅冷快堆（LFR）</td>
<td style="text-align:center">快</td>
<td style="text-align:center">铅</td>
<td style="text-align:center">480-800</td>
<td style="text-align:center">闭式</td>
</tr>
<tr>
<td style="text-align:center">气冷快堆（GFR）</td>
<td style="text-align:center">快</td>
<td style="text-align:center">氦气</td>
<td style="text-align:center">850</td>
<td style="text-align:center">闭式</td>
</tr>
</tbody>
</table>
<blockquote>
<p>SINAP——《核能科学与技术概论》课程</p>
</blockquote>
<h2 id="参考文献"> 参考文献</h2>
<p>[1] 杜祥琬,叶奇蓁,徐銤,万元熙,彭先觉,苏罡,杨勇,高翔,师学明.核能技术方向研究及发展路线图[J].中国工程科学,2018,20(03):17-24.</p>
<p>[2] 成松柏等. 第四代核能系统与钠冷快堆概论[M]. 北京:国防工业出版社,2018.</p>
]]></content:encoded>
    </item>
    <item>
      <title>抽样分布</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/mathematics/%E6%A6%82%E7%8E%87%E8%AE%BA_2/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/mathematics/%E6%A6%82%E7%8E%87%E8%AE%BA_2/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">抽样分布</source>
      <description>理解二项分布</description>
      <category>Maths</category>
      <pubDate>Wed, 18 May 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="伯努利分布-bernoulli-distribution"> 伯努利分布（Bernoulli Distribution）</h2>
<p>也称为”0-1分布“。在一次事件中，随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>只取<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8389em;vertical-align:-0.1944em;"></span><span>0</span><span>,</span><span style="margin-right:0.1667em;"></span><span>1</span></span></span></span>两个值，并且相应的概率为：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span style="margin-right:0.02778em;">r</span><span>(</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>1</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>p</span><span>，</span><span style="margin-right:0.13889em;">P</span><span style="margin-right:0.02778em;">r</span><span>(</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>0</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7278em;vertical-align:-0.0833em;"></span><span>1</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span>p</span><span>，</span><span>0</span><span style="margin-right:0.2778em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7335em;vertical-align:-0.1944em;"></span><span>p</span><span style="margin-right:0.2778em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>1</span></span></span></span></span></p>
<p>称：随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>服从参数为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>p</span></span></span></span>的伯努利分布，令<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span style="margin-right:0.03588em;">q</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.7278em;vertical-align:-0.0833em;"></span><span>1</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>p</span></span></span></span>，可写为：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10764em;">f</span><span>(</span><span>x</span><span>∣</span><span>p</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:3em;vertical-align:-1.25em;"></span><span><span style="top:0em;"><span>{</span></span><span><span><span><span><span><span style="height:1.69em;"><span style="top:-3.69em;"><span style="height:3.008em;"></span><span><span><span>p</span><span><span><span><span style="height:0.6644em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>x</span></span></span></span></span></span></span></span><span><span style="margin-right:0.03588em;">q</span><span><span><span><span style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>1</span><span>−</span><span>x</span></span></span></span></span></span></span></span></span><span>,</span></span></span><span style="top:-2.25em;"><span style="height:3.008em;"></span><span><span>0</span><span>,</span></span></span></span><span>​</span></span><span><span style="height:1.19em;"><span></span></span></span></span></span><span style="width:1em;"></span><span><span><span><span style="height:1.69em;"><span style="top:-3.69em;"><span style="height:3.008em;"></span><span><span>x</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>0</span><span>,</span><span style="margin-right:0.1667em;"></span><span>1</span><span>;</span></span></span><span style="top:-2.25em;"><span style="height:3.008em;"></span><span><span>x</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span><span></span></span></span><span></span></span></span></span></span><span>=</span></span><span style="margin-right:0.2778em;"></span><span>0</span><span>,</span><span style="margin-right:0.1667em;"></span><span>1.</span></span></span></span><span>​</span></span><span><span style="height:1.19em;"><span></span></span></span></span></span></span></span><span></span></span></span></span></span></span></p>
<p><strong><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>重伯努利试验</strong></p>
<p>​	如果<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>个随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>独立同分布（independent and identically distributed,  <strong>i.i.d.</strong>），并且都服从参数为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>p</span></span></span></span>的伯努利分布，则随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>形成参数为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>p</span></span></span></span>的<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>重伯努利试验。</p>
<h2 id="二项分布-binomial-distribution"> 二项分布（Binomial distribution）</h2>
<p>指在只有两个结果的<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>次独立的伯努利试验中，所期望的结果出现次数的概率，记作<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>∼</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.05017em;">B</span><span>(</span><span>n</span><span>,</span><span style="margin-right:0.1667em;"></span><span>p</span><span>)</span></span></span></span>。<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>次试验中正好得到<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6944em;"></span><span style="margin-right:0.03148em;">k</span></span></span></span>次成功的概率由**概率质量函数（Probability Mass Function, PMF）**给出：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>{</span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.03148em;">k</span><span>}</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.1491em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.07153em;">C</span><span><span><span><span style="height:0.8991em;"><span style="top:-2.453em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.03148em;">k</span></span></span></span><span>​</span></span><span><span style="height:0.247em;"><span></span></span></span></span></span></span><span><span>p</span><span><span><span><span style="height:0.8991em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.03148em;">k</span></span></span></span></span></span></span></span><span>(</span><span>1</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.1491em;vertical-align:-0.25em;"></span><span>p</span><span><span>)</span><span><span><span><span style="height:0.8991em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>n</span><span>−</span><span style="margin-right:0.03148em;">k</span></span></span></span></span></span></span></span></span></span></span></span></span></p>
<blockquote>
<p>概率质量函数是离散随机变量在各特定取值上的概率。</p>
</blockquote>
<p>其中二项式系数：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.0961em;vertical-align:-0.247em;"></span><span><span style="margin-right:0.07153em;">C</span><span><span><span><span style="height:0.8491em;"><span style="top:-2.453em;margin-left:-0.0715em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.03148em;">k</span></span></span></span><span>​</span></span><span><span style="height:0.247em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.4001em;vertical-align:-0.52em;"></span><span><span></span><span><span><span><span style="height:0.8801em;"><span style="top:-2.655em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.03148em;">k</span><span>!</span><span>(</span><span>n</span><span>−</span><span style="margin-right:0.03148em;">k</span><span>)!</span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.394em;"><span style="height:3em;"></span><span><span><span>n</span><span>!</span></span></span></span></span><span>​</span></span><span><span style="height:0.52em;"><span></span></span></span></span></span><span></span></span></span></span></span></p>
<p>二项分布的期望值和方差：等于每次单独试验的期望和方差的和（独立的伯努利试验）</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>E</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>n</span><span>p</span></span><span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span><span>D</span></span></span><span>(</span><span style="margin-right:0.07847em;">X</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>n</span><span>p</span><span>(</span><span>1</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span>p</span><span>)</span></span></span></span></span></p>
<h2 id="正态分布-normal-distribution"> 正态分布（Normal distribution）</h2>
<p>也称为高斯分布（Gaussian Distribution），是常见的连续概率分布。</p>
<p>若随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>服从一个位置参数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>μ</span></span></span></span>、一个尺度参数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.03588em;">σ</span></span></span></span>的正态分布，记为：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>∼</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.1141em;vertical-align:-0.25em;"></span><span style="margin-right:0.10903em;">N</span><span>(</span><span>μ</span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03588em;">σ</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span></span></span></span></span></p>
<p>其<strong>概率密度函数</strong>（Probability density function,  PDF）为：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10764em;">f</span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.4411em;vertical-align:-0.95em;"></span><span><span></span><span><span><span><span style="height:1.3214em;"><span style="top:-2.2028em;"><span style="height:3em;"></span><span><span style="margin-right:0.03588em;">σ</span><span><span><span><span style="height:0.9072em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="padding-left:0.833em;"><span>2</span><span style="margin-right:0.03588em;">π</span></span></span><span style="top:-2.8672em;"><span style="height:3em;"></span><span style="min-width:0.853em;height:1.08em;"><svg xmlns="http://www.w3.org/2000/svg" width='400em' height='1.08em' viewBox='0 0 400000 1080' preserveAspectRatio='xMinYMin slice'><path d='M95,702
c-2.7,0,-7.17,-2.7,-13.5,-8c-5.8,-5.3,-9.5,-10,-9.5,-14
c0,-2,0.3,-3.3,1,-4c1.3,-2.7,23.83,-20.7,67.5,-54
c44.2,-33.3,65.8,-50.3,66.5,-51c1.3,-1.3,3,-2,5,-2c4.7,0,8.7,3.3,12,10
s173,378,173,378c0.7,0,35.3,-71,104,-213c68.7,-142,137.5,-285,206.5,-429
c69,-144,104.5,-217.7,106.5,-221
l0 -0
c5.3,-9.3,12,-14,20,-14
H400000v40H845.2724
s-225.272,467,-225.272,467s-235,486,-235,486c-2.7,4.7,-9,7,-19,7
c-6,0,-10,-1,-12,-3s-194,-422,-194,-422s-65,47,-65,47z
M834 80h400000v40h-400000z'/></svg></span></span></span><span>​</span></span><span><span style="height:0.1328em;"><span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.93em;"><span></span></span></span></span></span><span></span></span><span><span><span>exp</span></span></span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;"><span>[</span></span><span>−</span><span><span></span><span><span><span><span style="height:1.4911em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span>2</span><span><span style="margin-right:0.03588em;">σ</span><span><span><span><span style="height:0.7401em;"><span style="top:-2.989em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span>(</span><span>x</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span>μ</span><span><span>)</span><span><span><span><span style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="top:0em;"><span>]</span></span></span></span></span></span></span></p>
<blockquote>
<p>概率密度函数：描述连续型随机变量在某个确定的取值点附近的可能性大小的函数。区分PMF。</p>
<p><strong>累积分布函数</strong>（Cumulative Distribution Function）：是概率密度函数的积分，能完整描述一个实随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>的概率分布。</p>
<p>对于一维实随机变量<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>，假设<strong>累积分布函数</strong><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.13889em;">F</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span></span></span></span>，如果存在可测函数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.10764em;">f</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1076em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span></span></span></span>，满足：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:8.1801em;vertical-align:-3.8401em;"></span><span><span><span><span><span style="height:4.3401em;"><span style="top:-6.5248em;"><span style="height:3.599em;"></span><span></span></span><span style="top:-3.8402em;"><span style="height:3.599em;"></span><span></span></span><span style="top:-0.9709em;"><span style="height:3.599em;"></span><span></span></span></span><span>​</span></span><span><span style="height:3.8401em;"><span></span></span></span></span></span><span><span><span><span style="height:4.3401em;"><span style="top:-6.5248em;"><span style="height:3.599em;"></span><span><span></span><span>∀</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span>∞</span><span style="margin-right:0.2778em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span><span>a</span><span style="margin-right:0.2778em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span><span>∞</span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.13889em;">F</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span style="margin-right:0.44445em;position:relative;top:-0.0011em;">∫</span><span><span><span><span style="height:1.4143em;"><span style="top:-1.7881em;margin-left:-0.4445em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>−</span><span>∞</span></span></span></span><span style="top:-3.8129em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>a</span></span></span></span><span>​</span></span><span><span style="height:0.9703em;"><span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.10764em;">f</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1076em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span><span>d</span><span>x</span></span></span><span style="top:-3.8402em;"><span style="height:3.599em;"></span><span><span></span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.44445em;position:relative;top:-0.0011em;">∫</span><span><span><span><span style="height:1.4143em;"><span style="top:-1.7881em;margin-left:-0.4445em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>−</span><span>∞</span></span></span></span><span style="top:-3.8129em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>∞</span></span></span></span></span><span>​</span></span><span><span style="height:0.9703em;"><span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.10764em;">f</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1076em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>1</span></span></span><span style="top:-0.9709em;"><span style="height:3.599em;"></span><span><span></span><span>∀</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span>∞</span><span style="margin-right:0.2778em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span><span>a</span><span style="margin-right:0.2778em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span><span>b</span><span style="margin-right:0.2778em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span><span>∞</span><span>,</span><span style="margin-right:0.1667em;"></span><span>P</span><span>[</span><span>a</span><span style="margin-right:0.2778em;"></span><span>&lt;</span><span style="margin-right:0.2778em;"></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>≤</span><span style="margin-right:0.2778em;"></span><span>b</span><span>]</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span style="margin-right:0.13889em;">F</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>b</span><span>)</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span><span style="margin-right:0.13889em;">F</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>a</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span><span style="margin-right:0.44445em;position:relative;top:-0.0011em;">∫</span><span><span><span><span style="height:1.599em;"><span style="top:-1.7881em;margin-left:-0.4445em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>a</span></span></span></span><span style="top:-3.8129em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>b</span></span></span></span><span>​</span></span><span><span style="height:0.9119em;"><span></span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.10764em;">f</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1076em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span><span>d</span><span>x</span></span></span></span><span>​</span></span><span><span style="height:3.8401em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:4.3401em;"><span style="top:-6.5248em;"><span style="height:3.599em;"></span><span></span></span><span style="top:-3.8402em;"><span style="height:3.599em;"></span><span></span></span><span style="top:-0.9709em;"><span style="height:3.599em;"></span><span></span></span></span><span>​</span></span><span><span style="height:3.8401em;"><span></span></span></span></span></span></span></span></span></p>
<p>如果概率密度函数<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.10764em;">f</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1076em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span></span></span></span>在一点<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>x</span></span></span></span>上连续，那么累积函数可导，导数为：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.0272em;vertical-align:-0.2753em;"></span><span><span style="margin-right:0.13889em;">F</span><span><span><span><span style="height:0.7519em;"><span style="top:-2.4247em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>′</span></span></span></span></span><span>​</span></span><span><span style="height:0.2753em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span style="margin-right:0.10764em;">f</span><span><span><span><span style="height:0.3283em;"><span style="top:-2.55em;margin-left:-0.1076em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.07847em;">X</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>(</span><span>x</span><span>)</span></span></span></span>。</p>
</blockquote>
<p><strong>标准正态分布（standard normal distribution）</strong></p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2778em;"></span><span>∼</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.1141em;vertical-align:-0.25em;"></span><span style="margin-right:0.10903em;">N</span><span>(</span><span>μ</span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03588em;">σ</span><span><span><span><span style="height:0.8641em;"><span style="top:-3.113em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">Y</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.0463em;vertical-align:-0.686em;"></span><span><span></span><span><span><span><span style="height:1.3603em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span style="margin-right:0.03588em;">σ</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span style="margin-right:0.07847em;">X</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span>μ</span></span></span></span><span>​</span></span><span><span style="height:0.686em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.2778em;"></span><span>∼</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10903em;">N</span><span>(</span><span>0</span><span>,</span><span style="margin-right:0.1667em;"></span><span>1</span><span>)</span></span></span></span></span></p>
<p><strong>置信区间（the interval of confidence）</strong></p>
<p>对平均数进行多次抽样，根据中心极限定理会形成一个正态分布概率密度函数，此时得到一个平均值<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5678em;"></span><span><span><span><span style="height:0.5678em;"><span style="top:-3em;"><span style="height:3em;"></span><span>x</span></span><span style="top:-3em;"><span style="height:3em;"></span><span style="left:-0.2222em;"><span>ˉ</span></span></span></span></span></span></span></span></span></span>，标准正态分布的期望为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>μ</span></span></span></span>，方差为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.2334em;vertical-align:-0.538em;"></span><span><span></span><span><span><span><span style="height:0.6954em;"><span style="top:-2.6259em;"><span style="height:3em;"></span><span><span><span><span><span><span style="height:0.8059em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="padding-left:0.833em;"><span>n</span></span></span><span style="top:-2.7659em;"><span style="height:3em;"></span><span style="min-width:0.853em;height:1.08em;"><svg xmlns="http://www.w3.org/2000/svg" width='400em' height='1.08em' viewBox='0 0 400000 1080' preserveAspectRatio='xMinYMin slice'><path d='M95,702
c-2.7,0,-7.17,-2.7,-13.5,-8c-5.8,-5.3,-9.5,-10,-9.5,-14
c0,-2,0.3,-3.3,1,-4c1.3,-2.7,23.83,-20.7,67.5,-54
c44.2,-33.3,65.8,-50.3,66.5,-51c1.3,-1.3,3,-2,5,-2c4.7,0,8.7,3.3,12,10
s173,378,173,378c0.7,0,35.3,-71,104,-213c68.7,-142,137.5,-285,206.5,-429
c69,-144,104.5,-217.7,106.5,-221
l0 -0
c5.3,-9.3,12,-14,20,-14
H400000v40H845.2724
s-225.272,467,-225.272,467s-235,486,-235,486c-2.7,4.7,-9,7,-19,7
c-6,0,-10,-1,-12,-3s-194,-422,-194,-422s-65,47,-65,47z
M834 80h400000v40h-400000z'/></svg></span></span></span><span>​</span></span><span><span style="height:0.2341em;"><span></span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.394em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.03588em;">σ</span></span></span></span></span><span>​</span></span><span><span style="height:0.538em;"><span></span></span></span></span></span><span></span></span></span></span></span>。</p>
<p>有95%的数据集中在<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.2334em;vertical-align:-0.538em;"></span><span>−</span><span>2</span><span><span></span><span><span><span><span style="height:0.6954em;"><span style="top:-2.6259em;"><span style="height:3em;"></span><span><span><span><span><span><span style="height:0.8059em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="padding-left:0.833em;"><span>n</span></span></span><span style="top:-2.7659em;"><span style="height:3em;"></span><span style="min-width:0.853em;height:1.08em;"><svg xmlns="http://www.w3.org/2000/svg" width='400em' height='1.08em' viewBox='0 0 400000 1080' preserveAspectRatio='xMinYMin slice'><path d='M95,702
c-2.7,0,-7.17,-2.7,-13.5,-8c-5.8,-5.3,-9.5,-10,-9.5,-14
c0,-2,0.3,-3.3,1,-4c1.3,-2.7,23.83,-20.7,67.5,-54
c44.2,-33.3,65.8,-50.3,66.5,-51c1.3,-1.3,3,-2,5,-2c4.7,0,8.7,3.3,12,10
s173,378,173,378c0.7,0,35.3,-71,104,-213c68.7,-142,137.5,-285,206.5,-429
c69,-144,104.5,-217.7,106.5,-221
l0 -0
c5.3,-9.3,12,-14,20,-14
H400000v40H845.2724
s-225.272,467,-225.272,467s-235,486,-235,486c-2.7,4.7,-9,7,-19,7
c-6,0,-10,-1,-12,-3s-194,-422,-194,-422s-65,47,-65,47z
M834 80h400000v40h-400000z'/></svg></span></span></span><span>​</span></span><span><span style="height:0.2341em;"><span></span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.394em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.03588em;">σ</span></span></span></span></span><span>​</span></span><span><span style="height:0.538em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.2778em;"></span><span>≤</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8304em;vertical-align:-0.1944em;"></span><span>μ</span><span style="margin-right:0.2778em;"></span><span>≤</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1.2334em;vertical-align:-0.538em;"></span><span>2</span><span><span></span><span><span><span><span style="height:0.6954em;"><span style="top:-2.6259em;"><span style="height:3em;"></span><span><span><span><span><span><span style="height:0.8059em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="padding-left:0.833em;"><span>n</span></span></span><span style="top:-2.7659em;"><span style="height:3em;"></span><span style="min-width:0.853em;height:1.08em;"><svg xmlns="http://www.w3.org/2000/svg" width='400em' height='1.08em' viewBox='0 0 400000 1080' preserveAspectRatio='xMinYMin slice'><path d='M95,702
c-2.7,0,-7.17,-2.7,-13.5,-8c-5.8,-5.3,-9.5,-10,-9.5,-14
c0,-2,0.3,-3.3,1,-4c1.3,-2.7,23.83,-20.7,67.5,-54
c44.2,-33.3,65.8,-50.3,66.5,-51c1.3,-1.3,3,-2,5,-2c4.7,0,8.7,3.3,12,10
s173,378,173,378c0.7,0,35.3,-71,104,-213c68.7,-142,137.5,-285,206.5,-429
c69,-144,104.5,-217.7,106.5,-221
l0 -0
c5.3,-9.3,12,-14,20,-14
H400000v40H845.2724
s-225.272,467,-225.272,467s-235,486,-235,486c-2.7,4.7,-9,7,-19,7
c-6,0,-10,-1,-12,-3s-194,-422,-194,-422s-65,47,-65,47z
M834 80h400000v40h-400000z'/></svg></span></span></span><span>​</span></span><span><span style="height:0.2341em;"><span></span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.394em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.03588em;">σ</span></span></span></span></span><span>​</span></span><span><span style="height:0.538em;"><span></span></span></span></span></span><span></span></span></span></span></span>范围内，即平均值<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5678em;"></span><span><span><span><span style="height:0.5678em;"><span style="top:-3em;"><span style="height:3em;"></span><span>x</span></span><span style="top:-3em;"><span style="height:3em;"></span><span style="left:-0.2222em;"><span>ˉ</span></span></span></span></span></span></span></span></span></span>有95%的概率落在<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7804em;vertical-align:-0.136em;"></span><span>−</span><span>2</span><span style="margin-right:0.03588em;">σ</span><span style="margin-right:0.2778em;"></span><span>≤</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8304em;vertical-align:-0.1944em;"></span><span>μ</span><span style="margin-right:0.2778em;"></span><span>≤</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>2</span><span style="margin-right:0.03588em;">σ</span></span></span></span>范围内，即：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span><span><span><span style="height:0.5678em;"><span style="top:-3em;"><span style="height:3em;"></span><span>x</span></span><span style="top:-3em;"><span style="height:3em;"></span><span style="left:-0.2222em;"><span>ˉ</span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:2.0376em;vertical-align:-0.93em;"></span><span>2</span><span><span></span><span><span><span><span style="height:1.1076em;"><span style="top:-2.3097em;"><span style="height:3em;"></span><span><span><span><span><span style="height:0.8003em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="padding-left:0.833em;"><span>n</span></span></span><span style="top:-2.7603em;"><span style="height:3em;"></span><span style="min-width:0.853em;height:1.08em;"><svg xmlns="http://www.w3.org/2000/svg" width='400em' height='1.08em' viewBox='0 0 400000 1080' preserveAspectRatio='xMinYMin slice'><path d='M95,702
c-2.7,0,-7.17,-2.7,-13.5,-8c-5.8,-5.3,-9.5,-10,-9.5,-14
c0,-2,0.3,-3.3,1,-4c1.3,-2.7,23.83,-20.7,67.5,-54
c44.2,-33.3,65.8,-50.3,66.5,-51c1.3,-1.3,3,-2,5,-2c4.7,0,8.7,3.3,12,10
s173,378,173,378c0.7,0,35.3,-71,104,-213c68.7,-142,137.5,-285,206.5,-429
c69,-144,104.5,-217.7,106.5,-221
l0 -0
c5.3,-9.3,12,-14,20,-14
H400000v40H845.2724
s-225.272,467,-225.272,467s-235,486,-235,486c-2.7,4.7,-9,7,-19,7
c-6,0,-10,-1,-12,-3s-194,-422,-194,-422s-65,47,-65,47z
M834 80h400000v40h-400000z'/></svg></span></span></span><span>​</span></span><span><span style="height:0.2397em;"><span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span style="margin-right:0.03588em;">σ</span></span></span></span><span>​</span></span><span><span style="height:0.93em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.2778em;"></span><span>≤</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8304em;vertical-align:-0.1944em;"></span><span>μ</span><span style="margin-right:0.2778em;"></span><span>≤</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span><span><span><span style="height:0.5678em;"><span style="top:-3em;"><span style="height:3em;"></span><span>x</span></span><span style="top:-3em;"><span style="height:3em;"></span><span style="left:-0.2222em;"><span>ˉ</span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:2.0376em;vertical-align:-0.93em;"></span><span>2</span><span><span></span><span><span><span><span style="height:1.1076em;"><span style="top:-2.3097em;"><span style="height:3em;"></span><span><span><span><span><span style="height:0.8003em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="padding-left:0.833em;"><span>n</span></span></span><span style="top:-2.7603em;"><span style="height:3em;"></span><span style="min-width:0.853em;height:1.08em;"><svg xmlns="http://www.w3.org/2000/svg" width='400em' height='1.08em' viewBox='0 0 400000 1080' preserveAspectRatio='xMinYMin slice'><path d='M95,702
c-2.7,0,-7.17,-2.7,-13.5,-8c-5.8,-5.3,-9.5,-10,-9.5,-14
c0,-2,0.3,-3.3,1,-4c1.3,-2.7,23.83,-20.7,67.5,-54
c44.2,-33.3,65.8,-50.3,66.5,-51c1.3,-1.3,3,-2,5,-2c4.7,0,8.7,3.3,12,10
s173,378,173,378c0.7,0,35.3,-71,104,-213c68.7,-142,137.5,-285,206.5,-429
c69,-144,104.5,-217.7,106.5,-221
l0 -0
c5.3,-9.3,12,-14,20,-14
H400000v40H845.2724
s-225.272,467,-225.272,467s-235,486,-235,486c-2.7,4.7,-9,7,-19,7
c-6,0,-10,-1,-12,-3s-194,-422,-194,-422s-65,47,-65,47z
M834 80h400000v40h-400000z'/></svg></span></span></span><span>​</span></span><span><span style="height:0.2397em;"><span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span style="margin-right:0.03588em;">σ</span></span></span></span><span>​</span></span><span><span style="height:0.93em;"><span></span></span></span></span></span><span></span></span></span></span></span></span></p>
<p>是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>μ</span></span></span></span>的置信水平为95%的置信区间，一般用<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7278em;vertical-align:-0.0833em;"></span><span>1</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.4306em;"></span><span style="margin-right:0.0037em;">α</span></span></span></span>表示<strong>置信水平（the level of confidence）</strong>，即上述<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span style="margin-right:0.0037em;">α</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0.05</span></span></span></span>。记作：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:4.2em;vertical-align:-1.85em;"></span><span><span><span><span><span style="height:2.35em;"><span style="top:-4.35em;"><span style="height:3.45em;"></span><span><span style="margin-right:0.13889em;">P</span><span style="margin-right:0.1667em;"></span><span><span style="top:0em;"><span>{</span></span><span><span><span><span style="height:0.5678em;"><span style="top:-3em;"><span style="height:3em;"></span><span>x</span></span><span style="top:-3em;"><span style="height:3em;"></span><span style="left:-0.2222em;"><span>ˉ</span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span><span style="margin-right:0.04398em;">z</span><span><span><span><span style="height:0.3341em;"><span style="top:-2.85em;margin-left:-0.044em;margin-right:0.05em;"><span style="height:3em;"></span><span><span><span><span></span><span><span><span><span style="height:0.6915em;"><span style="top:-2.656em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.04398em;">z</span></span></span></span><span style="top:-3.2255em;"><span style="height:3em;"></span><span style="border-bottom-width:0.049em;"></span></span><span style="top:-3.384em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.0037em;">α</span></span></span></span></span><span>​</span></span><span><span style="height:0.344em;"><span></span></span></span></span></span><span></span></span></span></span></span></span><span>​</span></span><span><span style="height:0.3908em;"><span></span></span></span></span></span></span><span><span></span><span><span><span><span style="height:1.1076em;"><span style="top:-2.3097em;"><span style="height:3em;"></span><span><span><span><span><span style="height:0.8003em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="padding-left:0.833em;"><span>n</span></span></span><span style="top:-2.7603em;"><span style="height:3em;"></span><span style="min-width:0.853em;height:1.08em;"><svg xmlns="http://www.w3.org/2000/svg" width='400em' height='1.08em' viewBox='0 0 400000 1080' preserveAspectRatio='xMinYMin slice'><path d='M95,702
c-2.7,0,-7.17,-2.7,-13.5,-8c-5.8,-5.3,-9.5,-10,-9.5,-14
c0,-2,0.3,-3.3,1,-4c1.3,-2.7,23.83,-20.7,67.5,-54
c44.2,-33.3,65.8,-50.3,66.5,-51c1.3,-1.3,3,-2,5,-2c4.7,0,8.7,3.3,12,10
s173,378,173,378c0.7,0,35.3,-71,104,-213c68.7,-142,137.5,-285,206.5,-429
c69,-144,104.5,-217.7,106.5,-221
l0 -0
c5.3,-9.3,12,-14,20,-14
H400000v40H845.2724
s-225.272,467,-225.272,467s-235,486,-235,486c-2.7,4.7,-9,7,-19,7
c-6,0,-10,-1,-12,-3s-194,-422,-194,-422s-65,47,-65,47z
M834 80h400000v40h-400000z'/></svg></span></span></span><span>​</span></span><span><span style="height:0.2397em;"><span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span style="margin-right:0.03588em;">σ</span></span></span></span><span>​</span></span><span><span style="height:0.93em;"><span></span></span></span></span></span><span></span></span><span style="margin-right:0.2778em;"></span><span>≤</span><span style="margin-right:0.2778em;"></span><span>μ</span><span style="margin-right:0.2778em;"></span><span>≤</span><span style="margin-right:0.2778em;"></span><span><span><span><span style="height:0.5678em;"><span style="top:-3em;"><span style="height:3em;"></span><span>x</span></span><span style="top:-3em;"><span style="height:3em;"></span><span style="left:-0.2222em;"><span>ˉ</span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span><span><span style="margin-right:0.04398em;">z</span><span><span><span><span style="height:0.3341em;"><span style="top:-2.85em;margin-left:-0.044em;margin-right:0.05em;"><span style="height:3em;"></span><span><span><span><span></span><span><span><span><span style="height:0.6915em;"><span style="top:-2.656em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.04398em;">z</span></span></span></span><span style="top:-3.2255em;"><span style="height:3em;"></span><span style="border-bottom-width:0.049em;"></span></span><span style="top:-3.384em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.0037em;">α</span></span></span></span></span><span>​</span></span><span><span style="height:0.344em;"><span></span></span></span></span></span><span></span></span></span></span></span></span><span>​</span></span><span><span style="height:0.3908em;"><span></span></span></span></span></span></span><span><span></span><span><span><span><span style="height:1.1076em;"><span style="top:-2.3097em;"><span style="height:3em;"></span><span><span><span><span><span style="height:0.8003em;"><span style="top:-3em;"><span style="height:3em;"></span><span style="padding-left:0.833em;"><span>n</span></span></span><span style="top:-2.7603em;"><span style="height:3em;"></span><span style="min-width:0.853em;height:1.08em;"><svg xmlns="http://www.w3.org/2000/svg" width='400em' height='1.08em' viewBox='0 0 400000 1080' preserveAspectRatio='xMinYMin slice'><path d='M95,702
c-2.7,0,-7.17,-2.7,-13.5,-8c-5.8,-5.3,-9.5,-10,-9.5,-14
c0,-2,0.3,-3.3,1,-4c1.3,-2.7,23.83,-20.7,67.5,-54
c44.2,-33.3,65.8,-50.3,66.5,-51c1.3,-1.3,3,-2,5,-2c4.7,0,8.7,3.3,12,10
s173,378,173,378c0.7,0,35.3,-71,104,-213c68.7,-142,137.5,-285,206.5,-429
c69,-144,104.5,-217.7,106.5,-221
l0 -0
c5.3,-9.3,12,-14,20,-14
H400000v40H845.2724
s-225.272,467,-225.272,467s-235,486,-235,486c-2.7,4.7,-9,7,-19,7
c-6,0,-10,-1,-12,-3s-194,-422,-194,-422s-65,47,-65,47z
M834 80h400000v40h-400000z'/></svg></span></span></span><span>​</span></span><span><span style="height:0.2397em;"><span></span></span></span></span></span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span style="margin-right:0.03588em;">σ</span></span></span></span><span>​</span></span><span><span style="height:0.93em;"><span></span></span></span></span></span><span></span></span><span style="top:0em;"><span>}</span></span></span></span></span><span style="top:-2.26em;"><span style="height:3.45em;"></span><span><span style="margin-right:0.0037em;">α</span></span></span></span><span>​</span></span><span><span style="height:1.85em;"><span></span></span></span></span></span><span><span><span><span style="height:2.35em;"><span style="top:-4.35em;"><span style="height:3.45em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>1</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span><span style="margin-right:0.0037em;">α</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>0.95</span></span></span><span style="top:-2.26em;"><span style="height:3.45em;"></span><span><span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span><span>0.05</span></span></span></span><span>​</span></span><span><span style="height:1.85em;"><span></span></span></span></span></span></span></span><span><span><span><span style="height:2.35em;"><span style="top:-4.35em;"><span style="height:3.45em;"></span><span></span></span><span style="top:-2.26em;"><span style="height:3.45em;"></span><span></span></span></span><span>​</span></span><span><span style="height:1.85em;"><span></span></span></span></span></span></span></span></span></p>
<p>可以查找对应的：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.04398em;">z</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.044em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>0.025</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>1.96</span></span></span></span>。</p>
<p>落在该置信区间的置信水平为95%，该值也称为<strong>覆盖率（coverage probability ）</strong>。</p>
]]></content:encoded>
    </item>
    <item>
      <title>概率论-贝叶斯公式</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/mathematics/%E6%A6%82%E7%8E%87%E8%AE%BA_1/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/mathematics/%E6%A6%82%E7%8E%87%E8%AE%BA_1/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">概率论-贝叶斯公式</source>
      <description>贝叶斯公式。</description>
      <category>Maths</category>
      <pubDate>Mon, 18 Apr 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<h2 id="条件概率-conditional-probability"> 条件概率（Conditional Probability）</h2>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span>A</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.05017em;">B</span></span></span></span>是样本空间<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>Ω</span></span></span></span>中的两个时间，其中<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2778em;"></span><span>&gt;</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span>，在事件<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">B</span></span></span></span>发生的条件下，事件<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>A</span></span></span></span>发生的概率为：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span>A</span><span>∣</span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.363em;vertical-align:-0.936em;"></span><span><span></span><span><span><span><span style="height:1.427em;"><span style="top:-2.314em;"><span style="height:3em;"></span><span><span style="margin-right:0.13889em;">P</span><span>(</span><span style="margin-right:0.05017em;">B</span><span>)</span></span></span><span style="top:-3.23em;"><span style="height:3em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-3.677em;"><span style="height:3em;"></span><span><span style="margin-right:0.13889em;">P</span><span>(</span><span>A</span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span><span style="margin-right:0.05017em;">B</span><span>)</span></span></span></span><span>​</span></span><span><span style="height:0.936em;"><span></span></span></span></span></span><span></span></span></span></span></span></span></p>
<p>乘法公式（乘法定理）：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span>A</span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span>A</span><span>∣</span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.13889em;">P</span><span>(</span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span style="margin-right:0.05017em;">B</span><span>∣</span><span>A</span><span>)</span><span style="margin-right:0.13889em;">P</span><span>(</span><span>A</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span>A</span><span>)</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2778em;"></span><span><span><span><span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span><span></span></span></span><span></span></span></span></span></span><span>=</span></span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6444em;"></span><span>0</span></span></span></span></span></p>
<p>加法公式：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span><span>A</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>∪</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>A</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span><span>A</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span><span>A</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2222em;"></span><span>−</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span><span>A</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>∩</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span><span>A</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span style="margin-right:0.05017em;">B</span><span>)</span></span></span></span></span></p>
<p>全概率公式：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.9291em;vertical-align:-1.2777em;"></span><span><span><span><span style="height:1.6514em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span><span>A</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.13889em;">P</span><span>(</span><span style="margin-right:0.05017em;">B</span><span>∣</span><span><span>A</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span></span></span></span></span></p>
<h2 id="贝叶斯定理-bayes-theorem"> 贝叶斯定理（Bayes' Theorem）</h2>
<p>若事件<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">B</span></span></span></span>只能与事件<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.0087em;vertical-align:-0.2587em;"></span><span>{</span><span><span>A</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span>}</span><span><span><span><span style="height:0.6644em;"><span style="top:-2.4413em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.2587em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>n</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8389em;vertical-align:-0.1944em;"></span><span>1</span><span>,</span><span style="margin-right:0.1667em;"></span><span>2</span><span>,</span><span style="margin-right:0.1667em;"></span><span>...</span></span></span></span>中之一同时发生，则有<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.05017em;">B</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:2.3291em;vertical-align:-0.9777em;"></span><span><span><span><span style="height:1.3514em;"><span style="top:-2.1223em;margin-left:0em;"><span style="height:3em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>∑</span></span></span><span style="top:-3.95em;margin-left:0em;"><span style="height:3em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.9777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span>A</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.05017em;">B</span></span></span></span>，由乘法公式和全概率公式，有：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span><span>A</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span style="margin-right:0.05017em;">B</span><span>)</span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:3.7822em;vertical-align:-2.3552em;"></span><span><span></span><span><span><span><span style="height:1.427em;"><span style="top:-2.11em;"><span style="height:3.3514em;"></span><span><span><span><span><span style="height:1.3514em;"><span style="top:-2.1223em;margin-left:0em;"><span style="height:3em;"></span><span><span><span style="margin-right:0.05724em;">j</span><span>=</span><span>1</span></span></span></span><span style="top:-3em;"><span style="height:3em;"></span><span><span>∑</span></span></span><span style="top:-3.95em;margin-left:0em;"><span style="height:3em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:1.1138em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span><span>A</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.13889em;">P</span><span>(</span><span style="margin-right:0.05017em;">B</span><span>∣</span><span><span>A</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.05724em;">j</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span>)</span></span></span><span style="top:-3.5814em;"><span style="height:3.3514em;"></span><span style="border-bottom-width:0.04em;"></span></span><span style="top:-4.0284em;"><span style="height:3.3514em;"></span><span><span style="margin-right:0.13889em;">P</span><span>(</span><span><span>A</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span><span style="margin-right:0.13889em;">P</span><span>(</span><span style="margin-right:0.05017em;">B</span><span>∣</span><span><span>A</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span></span></span></span><span>​</span></span><span><span style="height:2.3552em;"><span></span></span></span></span></span><span></span></span></span></span></span></span></p>
<p>其中：</p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span><span>A</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span></span></span></span>是<strong>先验概率（prior probability）</strong>，反映”原因“发生的可能性大小；</p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span><span>A</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>∣</span><span style="margin-right:0.05017em;">B</span><span>)</span></span></span></span>是<strong>似然概率（likelihood probability）</strong>，反映”原因“发生情况下，试验发生的可能性大小；</p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.13889em;">P</span><span>(</span><span style="margin-right:0.05017em;">B</span><span>∣</span><span><span>A</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span></span></span></span>是<strong>后验概率（Posterior probability）</strong>，反映在试验发生的情况下对”原因“发生概率进行考究知识。</p>
<p><a href="https://www.bilibili.com/video/BV1R7411a76r" target="_blank" rel="noopener noreferrer">贝叶斯定理-概率论直觉化-from-3Blue1Brown</a></p>
]]></content:encoded>
    </item>
    <item>
      <title>HTML基础知识</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/web/HTML%E6%95%99%E7%A8%8B/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/web/HTML%E6%95%99%E7%A8%8B/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">HTML基础知识</source>
      <description>整理的关于Web前端平台搭建涉及技术的基础知识——HTML部分。</description>
      <category>ComputerScience</category>
      <pubDate>Thu, 17 Feb 2022 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<p>HTML（Hyper Text Markup Language）超文本标记语言，是一种结构化Web网页及其内容的标记语言。</p>
<p>实例：</p>
<div><pre><code><span><span>&lt;!</span><span>DOCTYPE</span> <span>html</span><span>></span></span>   <span>&lt;!--声明文档的类型 --></span>
<span><span><span>&lt;</span>html</span><span>></span></span>    <span>&lt;!--HTML的根元素，结束标志&lt;/html> --></span>
    <span><span><span>&lt;</span>head</span><span>></span></span>   <span>&lt;!--包含文档的元数据（metadata），如&lt;meta charset="utf-8">定义网页编码格式 --></span>
        <span><span><span>&lt;</span>title</span><span>></span></span>页面标题(w3cschool.cn)<span><span><span>&lt;/</span>title</span><span>></span></span>  <span>&lt;!--定义文档的标题 --></span>
    <span><span><span>&lt;/</span>head</span><span>></span></span>
    <span><span><span>&lt;</span>body</span><span>></span></span>                <span>&lt;!--定义文档的主体，网页可见的页面内容，有结束标志 --></span>
        <span><span><span>&lt;</span>h1</span><span>></span></span>我的第一个标题<span><span><span>&lt;/</span>h1</span><span>></span></span> <span>&lt;!--作为一个标题使用 --></span>
        <span><span><span>&lt;</span>p</span><span>></span></span>我的第一个段落。<span><span><span>&lt;/</span>p</span><span>></span></span> <span>&lt;!--作为一个段落显示 --></span>
    <span><span><span>&lt;/</span>body</span><span>></span></span>
<span><span><span>&lt;/</span>html</span><span>></span></span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br><span>6</span><br><span>7</span><br><span>8</span><br><span>9</span><br><span>10</span><br></div></div><table>
<thead>
<tr>
<th>标签</th>
<th>描述</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>&lt;html&gt;</code></td>
<td>定义HTML文档</td>
</tr>
<tr>
<td><code>&lt;body</code></td>
<td>定义文档的主体</td>
</tr>
<tr>
<td><code>&lt;h1&gt;-&lt;h6&gt;</code></td>
<td>定义html的标题</td>
</tr>
<tr>
<td><code>&lt;hr&gt;</code></td>
<td>定义水平线</td>
</tr>
<tr>
<td><code>&lt;!--... --&gt;</code></td>
<td>定义注释</td>
</tr>
</tbody>
</table>
<h2 id="p标签"> p标签</h2>
<h2 id="标题-h1"> 标题-h1</h2>
<p><code>&lt;h1&gt;</code>定义最大的标题（主标题），<code>&lt;h6&gt;</code>定义最小的标题</p>
<center><img src="./images/html标题.png" width=80% title="html标题"></center>
<p>浏览器会自动在标题的前后添加空行。</p>
<h2 id="水平线-hr"> 水平线-hr</h2>
<center><img src="./images/html水平线.png" width=80% title="html水平线"></center>
<h2 id="div标签"> div标签</h2>
<p><code>&lt;div&gt;</code>标签本身不代表任何东西，用于标记区域，网页中表示块、区域的意思。如样式化（使用class和id属性），用不同的语言（lang属性）标记html文档的某个部分或一个分隔区块。</p>
<p><code>&lt;div&gt;</code>常用于组合块级元素，便通过CSS来对这些元素进行格式化。</p>
<p><code>&lt;div&gt;</code>元素通常与CSS一起使用，布局网页</p>
<div><pre><code><span>&lt;!--文档中的一个区域显示为蓝色--></span>
<span><span><span>&lt;</span>div</span> <span><span>style</span><span><span>=</span><span>"</span><span><span>color</span><span>:</span>#0000FF</span><span>"</span></span></span><span>></span></span>
  <span><span><span>&lt;</span>h3</span><span>></span></span>这是一个在 div 元素中的标题。<span><span><span>&lt;/</span>h3</span><span>></span></span>
  <span><span><span>&lt;</span>p</span><span>></span></span>这是一个在 div 元素中的文本。<span><span><span>&lt;/</span>p</span><span>></span></span>
<span><span><span>&lt;/</span>div</span><span>></span></span>
</code></pre>
<div><span>1</span><br><span>2</span><br><span>3</span><br><span>4</span><br><span>5</span><br></div></div><h2 id="class属性"> class属性</h2>
<p>class属性定义元素的类名，指向样式表的类。html中类名是区分大小写的。</p>
<p>class给块添加一些属性，并可以给属性起名字，class=xxx。</p>
<center><img src="./images/html的class属性.png" width=80% title="html的class属性"></center>
<h2 id="id属性"> id属性</h2>
<p>id属性规定html元素的唯一的id，在html文档中必须是唯一的。id属性可用作链接锚（link anchor），通过JS或者CSS为带有指定id的元素改变或添加样式。</p>
<center><img src="./images/html的id属性.png" width=80% title="html的id属性"></center>
<h2 id="id和class对比"> id和class对比</h2>
<p>id具有唯一性，在一个网页中同一个命名只能使用一次。</p>
<p>class命名的类可以在一个网页中使用无数次。</p>
<p>在css样式表中书写时，<strong>id选择符前面应该加前缀符号‘#’，而class选择符前面应该加前缀符号‘.’</strong></p>
<p><strong>class作为一个样式，先定义好，可以应用到任何结构和内容上（可复用）；id作为元素的标签，用于区分不同结构和内容（像名字），id是先找到结构/内容，在找它定义的样式；</strong></p>
<p>class更多应用在文字版块和页面修饰中，id更多应用在布局和设计包含块、包含框中。</p>
<h2 id="span标签"> span标签</h2>
<p><code>&lt;span&gt;</code>元素是无语义的行内元素，对元素进行分组，以不同的样式显示。</p>
<p><code>&lt;span&gt;</code>元素可以应用class或id属性，class用于元素组（一类的元素），id用于标识单独唯一的元素。</p>
<p><code>&lt;span&gt;</code>包含的文本，可以使用CSS对它定义样式，或者使用JS进行操作。</p>
<div><pre><code><span><span><span>&lt;</span>p</span><span>></span></span>我有 <span><span><span>&lt;</span>span</span> <span><span>style</span><span><span>=</span><span>"</span><span><span>color</span><span>:</span>blue<span>;</span><span>font-weight</span><span>:</span>bold</span><span>"</span></span></span><span>></span></span>蓝色<span><span><span>&lt;/</span>span</span><span>></span></span> 的眼睛。<span><span><span>&lt;/</span>p</span><span>></span></span>
</code></pre>
<div><span>1</span><br></div></div><h2 id="style标签"> style标签</h2>
<p><code>&lt;style&gt;</code>标签包含了html文档的样式信息，默认在该元素内写入的样式指令是CSS。</p>
<p>一个html文档包含多个<code>&lt;style&gt;</code>标签。</p>
<p>需要链接外部样式表，使用<code>&lt;link&gt;</code>标签。</p>
<center><img src="./images/html的span标签.png" width=80% title="html的span标签"></center>
]]></content:encoded>
    </item>
    <item>
      <title>Web前端平台搭建</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/web/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/web/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Web前端平台搭建</source>
      <description>关于Web前端平台搭建技术学习整理。</description>
      <content:encoded><![CDATA[<blockquote>
<p>HTML+CSS+JavaScript配合使用</p>
</blockquote>
<p>​		<strong>HTML用于标记内容（主体）</strong>，是超文本语言的简称，重在内容组织，是一种不严谨、简单的标识性语言。HTML利用各种标签将页面中的元素组织起来，告诉浏览器该如何显示其中的内容。</p>
<p>​		<strong>CSS用于修饰内容样式（美化）</strong>，是层叠样式表的简称，用于表现HTML页面中元素的展现和排版。</p>
<p>​		<strong>JavaScript用于交互</strong>，是一种脚本语言，可运行在客户端或服务器端。JavaScript的解释器是JS引擎，JS引擎是浏览器的一部分。JavaScript主要用于扩展文档交互能力，使静态HTML具有一定的交互行为（包括表单提交、动画特效、弹窗等）</p>
<p>三者关系：</p>
<p>HTML与CSS、JS是不同的技术，可以独立存在；</p>
<p>HTML一般需要CSS和JS来配合使用，否则单一HTML文档无论是功能还是展示上效果都不理想；</p>
<p>CSS一般是不能脱离HTML或XML的，如果CSS脱离了HTML和XML，那就没有存在的必要的；</p>
<p>JS可以脱离HTML和CSS而独立存在；</p>
<p>JS可以操作HTML和CSS。</p>
]]></content:encoded>
    </item>
    <item>
      <title>Deep leanring for time series forecasting a survey</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/%E6%97%B6%E5%BA%8F%E9%A2%84%E6%B5%8B%E8%AE%BA%E6%96%87/Deep%20leanring%20for%20time%20series%20forecasting%20a%20survey/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/%E6%97%B6%E5%BA%8F%E9%A2%84%E6%B5%8B%E8%AE%BA%E6%96%87/Deep%20leanring%20for%20time%20series%20forecasting%20a%20survey/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Deep leanring for time series forecasting a survey</source>
      <description>时序预测的深度学习方法综述</description>
      <category>PapersReading</category>
      <pubDate>Fri, 20 Oct 2023 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<blockquote>
<p>Torres J F, Hadjout D, Sebaa A, et al. Deep learning for time series forecasting: a survey[J]. Big Data, 2021, 9(1): 3-21.</p>
</blockquote>
<h2 id="time-series-definition"> Time series definition</h2>
<p>A sequence of values, chronologically ordered, and observed over time.
Not every time series can be modeled in this way, due to some of the following reasons:
(1)missing data
(2)outlying data
(3)collected at irregular time periods
If the data are collected irregularly, this should be accounted for in the model.</p>
<h2 id="time-series-components"> Time series components</h2>
<p>Time series are usually characterized by three components: <strong>trend, seasonality, and irregular components</strong>, also known as residuals.</p>
<h2 id="key-issues"> Key issues</h2>
<h3 id="irregular-component"> irregular component</h3>
<p>Real-world time series present a meaningful <strong>irregular component</strong> and are not stationary(均值和方差随时间变化), turning this component into the most challenging one to model.
For this reason, to make accurate predictions for them is extremely difficult.
Many classical forecasting methods try to decompose the target time series into trend, seasonality, and residual components and make predictions for all of them seperately.</p>
<h3 id="short-and-long-time-series-forecasting"> short-and  long time series forecasting</h3>
<p>Flexible nonparametric(非参数) models could be used, but this still assumes that the model structure will work over the whole period of the data, which is not always true.
A better approach consists of allowing the models to vary over time, including:
①adjusting a parametric model with time-varying parameters;
②adjusting a nonparametric model with a time-based kernel.</p>
<h2 id="methods"> Methods</h2>
<p>(1)statistical approaches
ARIMA model
(2)machine-learning algorithms
(3)<strong>deep-learning models</strong>
①Deep feed forward neural networks(DFFNNs)
②Recurrent neural networks(RNNs)
traditional feed-forward neural networks cannot take into account temporal dependency, and RNNs arise precisely to address this problem.
③Elman network(ENN)
ENN consists of a classical one-layer feed-forward network but the hidden layer is connected to a new layer, called context layer, using fixed weight equal to one.
④Long short-term memory(LSTM)
LSTM recurrent networks emerge to solve the vanishing gradient problem.
⑤Gated recurrent units(GRU)
GRU is a simplification of LSTMs due to the high computational cost of the LSTM networks.
⑥Bidirectional RNN(BRNN)
A BRNN can be seen as two RNNs together, where the different hidden units have two values, one computed by forward and another one by backward.
⑦Deep recurrenct neural network(DRNN)
A DRNN can be considered as an RNN with more than one layer, alse called stacked RNN.
⑧Convolutional neural network(CNN)
A variant of CNN, called temporal CNNs(TCNs) has emerged for data sequence, competing directly with DRNNs in terms of execution times and memory requirements.</p>
<h2 id="practical-aspects"> Practical aspects</h2>
<p>Implementation——deep-learning frameworks
Hyper-parameters——model parameters and optimization parameters
Hyper-parameter optimization——search strategies and libraries
Hardware performance——CPU/GPU/TPU/IPU</p>
<h2 id="applications"> Applications</h2>
]]></content:encoded>
    </item>
    <item>
      <title>Challenges and approached to time-series forecasting</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/%E6%97%B6%E5%BA%8F%E9%A2%84%E6%B5%8B%E8%AE%BA%E6%96%87/Challenges%20and%20approached%20to%20time-series%20forecasting/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/%E6%97%B6%E5%BA%8F%E9%A2%84%E6%B5%8B%E8%AE%BA%E6%96%87/Challenges%20and%20approached%20to%20time-series%20forecasting/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Challenges and approached to time-series forecasting</source>
      <description>时序预测方法及挑战</description>
      <category>PapersReading</category>
      <pubDate>Fri, 20 Oct 2023 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<blockquote>
<p>Jadon S, Milczek J K, Patankar A. Challenges and approaches to time-series forecasting in data center telemetry: A Survey[J]. arXiv preprint arXiv:2101.04224, 2021.</p>
</blockquote>
<h2 id="multi-period-forecasting-definition"> Multi-period forecasting definition</h2>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.7167em;vertical-align:-0.2861em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>n</span><span>+</span><span>1</span><span>,</span><span>n</span><span>+</span><span>2</span><span>,</span><span>…</span><span>,</span><span>n</span><span>+</span><span>m</span></span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10764em;">f</span><span>(</span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>0</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.3361em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.03148em;">k</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>;</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>0</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span></span></span></span></p>
<p>Single or Next period prediction:</p>
<p><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6389em;vertical-align:-0.2083em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>n</span><span>+</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.10764em;">f</span><span>(</span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>0</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.13889em;">T</span><span><span><span><span style="height:0.3361em;"><span style="top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span style="margin-right:0.03148em;">k</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>;</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>0</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>,</span><span style="margin-right:0.1667em;"></span><span>…</span><span style="margin-right:0.1667em;"></span><span>,</span><span style="margin-right:0.1667em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>n</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span>)</span></span></span></span></p>
<p>There are various approaches to generating forecasts for multiple time periods:
①fixed-length forecast
②arbitrary-length forecast
③single-point rolling prediction
④fixed multiple-point rolling prediction</p>
<h2 id="challenges-in-multi-period-forecasting"> Challenges in Multi-Period forecasting</h2>
<p>①Large number of metrics to forecast
②High data volume
③Minimal or no data for some metrics
④Non-linear and multi-variate interactions among metrics
⑤Length of forecasting horizon
⑥Consuming applications that require high performance and low latency</p>
<h2 id="forecasting-techniques"> Forecasting Techniques</h2>
<h3 id="state-space-models-smms"> State Space Models(SMMs)</h3>
<p><strong>Pros:</strong> SSMs are particularly well-suited for applications where the structure of the time series is well-understood as they <strong>explicitly incorporate structual assumptions</strong> in the model.</p>
<p><strong>Cons:</strong> SMMs cannot infer shared patterns from multiple data sets of similar time series as they have to be fitted separately on each time series.</p>
<h4 id="arima"> ARIMA</h4>
<p><strong>Pros:</strong> ARIMA(autoregressive integrated moving average，自回归积分移动平均) is a standard bench marking technique, and works with limited data and limited computing power.
<strong>Cons:</strong> It is an univariate(单变量) technique, and not suitable for distributions with fat tails or volatility(波动) clusters. Besides, it has theoretical limitations such as the erroe terms needs to have a random normal distribution.
There are some <strong>multivariate extensions to ARIMA</strong> technique including** VAR**(vector autoregressive models) , <strong>VARMA</strong>, and <strong>ARIMAX</strong>. However, there <strong>do not appea</strong>r to be many references of successful applications of the multivariate extensions in practice.</p>
<h4 id="ewma"> EWMA</h4>
<p><strong>Pros:</strong> EWMA(exponential weighted moving average，指数加权移动平均) is possible to implement as an online model and it natively supports arbitrary lenght forecasts(任意长度的预测).
<strong>Cons:</strong> Model may be too simple to represent real life scenarios.</p>
<h4 id="grach"> GRACH</h4>
<p>GRACH(generalized autoregressive conditional heteroskedastic model, 广义自回归条件异方差模型) is a more powerful model as it supports heteroskedastic processes.</p>
<h4 id="seasonal-trend-decomposition-predictor"> Seasonal Trend Decomposition Predictor</h4>
<p>STD Predictor is an extension of linear regression technique that incorporates seasonal trends.
<strong>Prons:</strong></p>
<ul>
<li>Very fast and easy to use</li>
<li>Supports calculations of confidence intervals</li>
<li>Supports multivariate forecasting</li>
<li>Supports an arbitrary length forecasts</li>
</ul>
<p><strong>Cons:</strong></p>
<ul>
<li>Very simple, cannot model complex relations</li>
<li>Confidence intervals are over-simplified</li>
</ul>
<h4 id="seasonal-trend-autoregressive-predictor"> Seasonal Trend Autoregressive Predictor</h4>
<p>STAR Predictor extends the STD model by incorporating an autoregressive component.
<strong>Prons:</strong></p>
<ul>
<li>Fast and easy to use</li>
<li>Supports basic confidence intervals</li>
<li>Extensible to multivariate forecasting</li>
</ul>
<p><strong>Cons:</strong></p>
<ul>
<li>More complex than STD predictor, prone to overfitting</li>
<li>Slower than STD predictor</li>
</ul>
<h3 id="deep-neural-networks-dnns"> Deep Neural Networks(DNNs)</h3>
<p><strong>Pros:</strong> DNNs make very few stuctural assumptions regarding the underlying process and yet can identity complex patterns within and across multiple time series.</p>
<p><strong>Cons:</strong> DNNs require significantly more data for training and suffer from computationally intensive and a lack of interpretability.</p>
<h4 id="recurrent-neural-networks"> Recurrent neural networks</h4>
<p><strong>Prons:</strong></p>
<ul>
<li>Ability to learn complex features of time-series</li>
</ul>
<p><strong>Cons:</strong></p>
<ul>
<li>Vanishing gradient problem</li>
<li>Fixed size input is required for training</li>
</ul>
<h4 id="long-short-term-memory-networks"> Long short term memory networks</h4>
<p>LSTMs are explicitly designed to avoid the vanishing gradient problem. The architecture inherently remembers information over a long sequence of the time series.
<strong>Cons:</strong></p>
<ul>
<li>Computational intensity of training</li>
<li>Require large amount of data to train than other models</li>
<li>Fixed size input is required for training</li>
</ul>
<h4 id="bi-directional-lstms"> Bi-directional LSTMs</h4>
<p>Bidirectional LSTMs use two hidden states combined you are able in any point in time to preserve information from both past and future, which can understand context better.
<strong>Prons:</strong></p>
<ul>
<li>Preserves information from future as well as past</li>
<li>Proven to perform better than uni-directional LSTM in learning time-series distribution</li>
</ul>
<p><strong>Cons:</strong></p>
<ul>
<li>Takes a long time to run</li>
<li>Requires large amount of data to train</li>
<li>Fixed input size is required for traning</li>
</ul>
<h3 id="probabilistic-models"> Probabilistic Models</h3>
<p>Probabilistic models use optimization approaches to learn the parameters instead of casting data distribution into another distribution.</p>
<h4 id="hidden-markov-models"> Hidden markov models</h4>
<p><strong>Prons:</strong></p>
<ul>
<li>Gives better predictions even with less amount of data</li>
<li>Faster to train and predict</li>
</ul>
<p><strong>Cons:</strong></p>
<ul>
<li>Not good for predicions in long run due to its assumption of dependence only on current state.</li>
</ul>
]]></content:encoded>
    </item>
    <item>
      <title>Homepage</title>
      <link>https://jiangxj.top/vuepress/vuepress/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">Homepage</source>
    </item>
    <item>
      <title>基于Transformers模型的时序预测（TSF）方法</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/%E6%97%B6%E5%BA%8F%E9%A2%84%E6%B5%8B%E8%AE%BA%E6%96%87/Time%20Series%20Forecasting/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/%E6%97%B6%E5%BA%8F%E9%A2%84%E6%B5%8B%E8%AE%BA%E6%96%87/Time%20Series%20Forecasting/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">基于Transformers模型的时序预测（TSF）方法</source>
      <description>基于Transformer的时序预测（TSF）方法。</description>
      <category>PapersReading</category>
      <pubDate>Sat, 01 Jul 2023 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<blockquote>
<p>参考：相关顶会论文</p>
</blockquote>
<h2 id="transformers在时序任务上的研究综述"> Transformers在时序任务上的研究综述</h2>
<p>论文从网络架构（network structure）和使用场景（applications）两方面对Transfomer在时序数据上的研究进行了综述。在<strong>网络结构</strong>方面，论文阐述了low-level（module层）和high-level（架构层）的创新研究。在<strong>应用</strong>场景上，论文对Transformer在处理主流的时序任务过程中的insights、strengths和limitations进行了阐述。此外，论文也对<strong>时序建模的性能评价指标</strong>进行了说明，包括robustness analysis、model size analysis和seasonal-trend decomposition analysis。最后，论文给出了应用Transformer处理时序任务的可能的<strong>未来研究方向</strong>，包括：归纳偏置、Transformer和GNN相结合、预训练Transfomer、架构层次的Transformer变体、结合神经网络架构搜索（Neural architecture search，<strong>NAS</strong>）的Transformer。</p>
<p>Transformer在自然语言处理、计算机视觉、文本处理等领域显示出了强大的性能，Transformer在建模序列数据中的长范围依赖和交互上具有明显优势，并成功应用在多项时序任务，包括：预测（forecasting）、异常检测（anomaly detection）和分类（classification）等。</p>
<p>对时序数据长期时间依赖和短期时间依赖性进行建模，并且同时能捕捉时序数据季节性特征仍然是一个挑战。</p>
<p>已有的关于应用深度学习方法处理时序型任务的综述主要集中在：预测、分类、异常检测和数据增强等。Transformer作为深度学习领域中新出现的一个分支，已经在时序领域产生重要影响。</p>
<p>论文总结了Transformer在网络结构方面的创新主要包括：（1）增加位置编码；（2）注意力模块修改；（3）架构层面创新，即Transformer的变体。Transformer应用方面主要包括预测、异常检测和分类三种任务。</p>
<center><img src="./Time Series Forecasting.assets/image1.png" title="Transformer在时序建模上的分类" width=60% ></center>
<p>【时序Transformers的网络设计】</p>
<p>与LSTM或RNN不同，Transformer没有递归和卷积，输入嵌入中加入的位置编码，对序列信息进行建模。</p>
<p><strong>（1）position encoding</strong></p>
<p>由于Transformer相互交换位置是等价的，但是顺序信息对于时间序列非常重要，因此对输入时间序列的位置进行编码具有重要意义。一种常见的设计是：将位置信息编码为向量，然后作为附加输入与输入时间序列一起注入模型。位置信息编码主要包括以下三种形式：</p>
<ul>
<li>
<p>vanilla positional encoding</p>
<p>手工制作，性能和适应性相对较差，不能充分利用时间序列数据的重要特征。</p>
</li>
<li>
<p>learnable positional encoding</p>
<p>学习嵌入具有更强的灵活性，能够适应特定的任务。如使用LSTM对位置嵌入进行编码，以便于更好地利用时序中的顺序排列信息。</p>
</li>
<li>
<p>timestamp encoding</p>
<p>针对现实场景中可访问的时间戳信息。</p>
</li>
</ul>
<p><strong>（2）attention module</strong></p>
<p>自注意力机制是Transformer的核心，它可以看作是一个<strong>全连接层</strong>，其权值是根据输入模式的两两相似度动态生成的。self-attention与完全连接的层共享相同的最大路径长度，但参数量要小很多，适合建模长期依赖关系，但其计算消耗非常大。原始的Transformer的时间和内存复杂度都是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.0641em;vertical-align:-0.25em;"></span><span style="margin-right:0.02778em;">O</span><span>(</span><span><span>L</span><span><span><span><span style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span></span></span></span>。一些工作关注如何减少Transformer的时间和内存复杂度。</p>
<p><strong>（3）architecture-level innovation</strong></p>
<p>架构层面创新主要聚焦于提升模型对信息的整合能力和改进计算消耗两方面。</p>
<p>【时序Transformers的应用】
<strong>（1）forecasting</strong></p>
<p>主要包括时间序列预测（如销量预测、股价预测、气温预测等）、时空预测（如交通领域）和事件预测（根据过去事件的历史信息来预测未来事件发生的时间和标志）。</p>
<p>时间序列预测典型的方法包括：Informer、AST、Autoformer、FEDformer、TFT、SSDNet、Pyraformer、Aliformer等。</p>
<p>spatio-temporal forecasting典型方法包括：Traffic Transformer、时空图Transformer等</p>
<p>Event forecasting典型方法包括：SAHP、Transformer Hawkes、ANDTT等。</p>
<p><strong>（2）anomaly detection</strong></p>
<p>时序异常检测是从正常的时间序列中识别异常事件的出现，在工业届和医疗领域有较多应用。</p>
<p><strong>（3）classification</strong></p>
<p>时间序列分类任务。其中，分类Transfomer通常采样简单的编码器结构，自注意力层执行表示学习，前馈层得到每个类别的概率。</p>
<p>典型方法包括：GTN等。</p>
<blockquote>
<p>Wen Q, Zhou T, Zhang C, et al. Transformers in time series: A survey[J]. arXiv preprint arXiv:2202.07125, 2022.</p>
</blockquote>
<h2 id="transformers在时序预测上的应用研究"> Transformers在时序预测上的应用研究</h2>
<h3 id="logsparse-transformer模型"> LogSparse Transformer模型</h3>
<p>时序预测（times series forecasting）在太阳能发电厂能源输出、电力消耗、交通拥堵等场景中有广泛应用。</p>
<p>传统的时序预测模型包括状态空间模型（state space models，SSMs）和自回归模型（autoregressive models）。设计该类模型用于单独的适应每个时间序列，并且需要借助专业知识来手动选择趋势、季节性及其它特征。这极大地限制了该类方法在现代大规模时序预测任务中的应用。</p>
<p>深度神经网络（deep neural networks）被提出用于解决上述问题，其中一个典型的模型是循环神经网络（recurrent neural network，RNN）。RNNs模型容易受到梯度消失和梯度爆炸（gradident vanishing and exploding）的影响，导致<strong>模型难训练</strong>。针对此提出了RNN的变体模型LSTM和GRU，但对于<strong>长期依赖性</strong>（long-term dependency）建模性能表现不佳，尤其是现实世界中长期和短期依赖性重复性并存。因此，对长期依赖性进行有效建模是解决该问题的关键步骤。</p>
<p>Transformer结合注意力机制在处理序列数据取得很大成功，Transformer能够很好的理解存在长期依赖性的时序数据中重复出现的特征模式。但原始的Transfomer存在一定问题，包括：（1）局部不可感知（locality-agnostics）；（2）内存瓶颈（memeory bottleneck）。</p>
<p>针对Transformer存在的问题，论文提出了：</p>
<p>（1）卷积自注意力机制（convolutional self-attention）用于捕捉时序局部上下文信息。</p>
<center><img src="./Time Series Forecasting.assets/image2.png" title="卷积自注意力机制" width=60% ></center>
<p>传统的Transformer中，是对每个点单独进行<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span>Q</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.07153em;">K</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.22222em;">V</span></span></span></span>的投影进行计算。会导致如图（a）中两个红点，尽管在时间序列上的特征不同（一个陡增，一个是缓趋势），但由于绝对值一样，得到的两个attention是很接近的。同理，（c）中框起来的两个区域，其局部特征是类似的，但由于绝对值不一样，计算得到的attention差异不大（理论应该是较大的）。</p>
<p>引入卷积自注意力，使用stride为1，kernel大小为k的因果卷积来计算<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span>Q</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.07153em;">K</span></span></span></span>。通过因果卷积，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8778em;vertical-align:-0.1944em;"></span><span>Q</span><span>,</span><span style="margin-right:0.1667em;"></span><span style="margin-right:0.07153em;">K</span></span></span></span>可以更好地知晓当前时刻的局部时间序列信息。传统的Transformer相当于k的值为1。</p>
<p>（2）LogSparse Transformer模型，实现了将传统Transformer空间和时间复杂度<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.0641em;vertical-align:-0.25em;"></span><span style="margin-right:0.02778em;">O</span><span>(</span><span><span>L</span><span><span><span><span style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span></span></span></span>降为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.0641em;vertical-align:-0.25em;"></span><span style="margin-right:0.02778em;">O</span><span>(</span><span>L</span><span>(</span><span style="margin-right:0.01968em;">l</span><span>o</span><span>gL</span><span><span>)</span><span><span><span><span style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span></span></span></span>。</p>
<blockquote>
<p>Li S, Jin X, Xuan Y, et al. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting[J]. Advances in neural information processing systems, 2019, 32.</p>
<p>https://blog.csdn.net/qq_40206371/article/details/126369053</p>
</blockquote>
<h3 id="informer模型"> Informer模型</h3>
<p>时间序列预测（time-series forecasting）任务场景包括银行交易量预测、电力系统用电量预测、云服务访问量预测等。如果能提前预测到未来一段时间内的访问量/用电量，就可以提前进行资源部署，防止访问量过大耗尽现有计算资源，拖垮服务。</p>
<p>随着预测序列长度增加，预测难度越来越大，对于长序列预测（long sequence time-series forecasting，LSTF）问题，需要模型具有较强的处理长距离依赖（long-range dependency）问题的能力。Transformer已经在LSTF任务上表现出可以增强预测能力的潜力，但同时也存在以下几个问题：</p>
<p>（1）quadratic computation of self-attention</p>
<p>self-attention的时间和空间复杂度都是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1.0641em;vertical-align:-0.25em;"></span><span style="margin-right:0.02778em;">O</span><span>(</span><span><span>L</span><span><span><span><span style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span></span></span></span>，其中<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span>L</span></span></span></span>是序列长度。</p>
<p>（2）high memory usage</p>
<p>堆叠的encoder/decoder内存消耗为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.02778em;">O</span><span>(</span><span style="margin-right:0.09618em;">J</span><span style="margin-right:0.2222em;"></span><span>⋅</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1.0641em;vertical-align:-0.25em;"></span><span><span>L</span><span><span><span><span style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span><span>)</span></span></span></span>，其中<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.09618em;">J</span></span></span></span>是堆叠数量，限制了模型在处理长序列输入的扩展性。</p>
<p>（3）inherent limitation of the encoder-decoder architecture</p>
<p>encoder-decoder结构在解码时是采用step-by-step形式进行推理，预测序列越长，预测需要耗时越长（和RNN类似）。</p>
<p>针对上面问题，论文提出的改进如下：</p>
<p>（1）提出ProbSparse self-attention机制，时间复杂度将为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.02778em;">O</span><span>(</span><span>L</span><span style="margin-right:0.2222em;"></span><span>⋅</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.01968em;">l</span><span>o</span><span>gL</span><span>)</span></span></span></span></p>
<p>（2）提出self-attention蒸馏机制来缩短每一层的输入序列长度，降低计算量和内存消耗。</p>
<p>（3）提出生成式decoder机制，在inference阶段时一步（one forward step）得到结果，将预测时间复杂度由<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.02778em;">O</span><span>(</span><span style="margin-right:0.10903em;">N</span><span>)</span></span></span></span>降到<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:1em;vertical-align:-0.25em;"></span><span style="margin-right:0.02778em;">O</span><span>(</span><span>1</span><span>)</span></span></span></span>。</p>
<blockquote>
<p>Zhou H, Zhang S, Peng J, et al. Informer: Beyond efficient transformer for long sequence time-series forecasting[C]//Proceedings of the AAAI conference on artificial intelligence. 2021, 35(12): 11106-11115.</p>
<p>https://zhuanlan.zhihu.com/p/355133560</p>
</blockquote>
<h3 id="fedformer模型"> FEDformer模型</h3>
<p>long-term series forecasting任务是需要根据现有的数据对未来做出较长时间段的预测。在部分场景中，模型的输出长度可能达到1000以上，覆盖若干个周期，这对于预测模型的精度和计算效率提出了更高的要求。此外，时序数据往往会受到<strong>分布偏移</strong>和<strong>噪声</strong>的影响，增加了预测的难度。</p>
<p>Transformers用于long-term series forecasting任务存在两个问题：（1）模型训练代价昂贵；（2）不能捕捉时序数据全局特征（global view）。</p>
<p>针对于上诉问题，论文提出了两种思路：（1）周期趋势项分解（seasonal-trend decomposition）降低输入输出的分布差异；（2）提出一种在频域应用注意力机制的Transformer模型，增加对噪声的鲁棒性。</p>
<blockquote>
<p>Zhou T, Ma Z, Wen Q, et al. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting[C]//International Conference on Machine Learning. PMLR, 2022: 27268-27286.</p>
<p>https://zhuanlan.zhihu.com/p/528131016</p>
</blockquote>
<h2 id="transformers在时序异常检测上的应用研究"> Transformers在时序异常检测上的应用研究</h2>
<h3 id="基于关联性差异的时序异常检测"> 基于关联性差异的时序异常检测</h3>
<p>论文关注无监督时序异常检测问题。传统的带有无监督范式的时序异常检测方法，包括LOF，OC-SVM，SVDD等，很少考虑时间信息，并且很难泛化应用到未知的真实场景中。</p>
<p>基于深度神经网络的时序异常检测方法，往往是通过类型循环网络（RNN）学习时序数据点级别（pointwise）的表征，进而依赖重建或预测误差进行判断是否是异常点。该类方法学习的点级别表征信息量较小，并且可能会被正常模式主导，使得异常值难以区分。此外，通过计算点与点之间的重构误差或者预测误差很难对时间上下文信息（temporal context）进行深刻描述。</p>
<p>基于显式关联建模（explicit association modeling）检测异常，包括向量自回归（vector autoregression，VAR）、状态空间建模等。此外，图卷积神经网络（GNN）也被用于学习多变量时序数据存储的信息，虽然图的表达能力更强，但仍然受限于单个时间点，这对于处理复杂时序数据来说是不够的。</p>
<p>基于子序列的方法通过计算子序列之间的相似性来检测异常，这在探索更广泛的时间上下文时，这些方法无法捕捉每个时间点与整个序列之间的细粒度（fine-grained）时间关联。</p>
<p>论文提出了anomaly transformer方法，通过使用<strong>关联差异</strong>（association discrepancy）来进行异常检测。应用Transformer模型处理时序数据，可以通过自注意力机制获取每一个时间点的时间关联，这可以提供更为丰富的时间上下文信息来表征时序数据动态模式，包括周期性和趋势性（这一部分称为<strong>series-association</strong>）。此外，正常模式数据在长时序数据中占据主导地位，异常数据少，很难对整个时序数据的异常点建立强相关性。由于时序数据的连续，使得异常数据邻近的时间点和异常点具有相似性（<strong>prior-association</strong>）。论文提出的<strong>关联差异</strong>概念是利用每一个时间点的prior-association和series-association的距离（distance）来度量。</p>
<blockquote>
<p>参考文献</p>
<p>Xu J, Wu H, Wang J, et al. Anomaly transformer: Time series anomaly detection with association discrepancy[J]. arXiv preprint arXiv:2110.02642, 2021.</p>
<p>https://zhuanlan.zhihu.com/p/470844801</p>
</blockquote>
<h2 id="transformers在ltsf任务中是否真的有效"> Transformers在LTSF任务中是否真的有效？</h2>
<p>时序预测（time series forecasting, TSF）广泛应用在交通流量估算、能源管理，金融投资等领域。TSF方法的发展历经传统的统计方法（如ARIMA），到机器学习（如GBRT），再到深度学习阶段。</p>
<p>鉴于Transformer在序列建模上的优越性，时间序列预测架构的Transformer系列研究不断发展，包括LogTrans（NeurIPS 2019） Informer（AAAI 2021 Best paper），Autoformer（NeurIPS 2021），Pyraformer（ICLR 2022 Oral），Triformer（IJCAI 2022），FEDformer（ICML 2022）等。</p>
<p>多头自注意机制（multi-head self-attention mechanism）在提取长序列中的语义相关性任务（如长文本中的单词或者2D图像的补丁）中具有显著优越性。</p>
<p>自注意力机制通过positional encoding来保存序列中的语义信息，但直接应用在时间序列预测任务中容易丢失temporal information。时间序列数据本身是纯数值型数据，基本没有逐点语义关联性（point-wise semantic correlations），语义信息缺乏。时间序列数据中更多的是关注一些列时间点数据的关联性（temporal relations）以及数据点的顺序（order），自注意力机制固有的排列不变性（permutation invariant）不可避免地会丢失时间信息。</p>
<p>因此，<strong>Transformers在long-term time series forecasing（LTSF）任务中有效性还有待验证</strong>。</p>
<p>现有的基于Transformers的LTSF解决方案已经证明了比传统方法在预测精度上有了很大提高，但实验对比的基线模型都是采用autoregressive（自回归）或iterated multi-step（<strong>IMS</strong>，迭代多步骤）的方法，这些方法存在的一个显著问题是<strong>误差累积</strong>。基于Transformers的LTSF还需要和直接多步（direct multi-step，<strong>DMS</strong>）预测方法进行比较以验证实际性能。</p>
<p>需要注意的是，不是所有的时间预测数据都是可预测的，LTSF只对那些具有相对明确的趋势和周期性的时间序列是可行的。</p>
<p>现有的Transformer-based方法处理LTSF任务采用的策略主要包括：</p>
<p><strong>（1）时序分解（time series decomposition）</strong>：常用的零均值归一化（normalization with zero-mean）；Autoformer提出季节性趋势分解（seasonal-trend decomposition）；FEDformer提出混合专家策略，设置多个大小的移动平均核来提取趋势分量。</p>
<p><strong>（2）输入嵌入策略（input embedding strategies）</strong>：Transformer中的自注意力层无法保存时序数据的位置信息，充分利用包括：时序数据的顺序（即local positional information）、周月年等分层时间戳（hierarchical timestamps）、假期活动等不可知时间戳（agnostic timestamps）等来增强模型时间序列输入的时间上下文（temporal context），通过在输入序列中嵌入多个embedding实现，包括：fixed positional encoding，channel projection embedding，learnable temporal embedding等。</p>
<p><strong>（3）自注意力机制模式（self-attention schemes）</strong>：LogTrans和Pyraformer引入稀疏性偏差（sparsity bias）提高模型有效性；Informer和FEDformer使用使用自注意力矩阵的低秩（low-rank）特性。</p>
<p><strong>（4）解码器（Decoders）</strong>：原版的Transformer解码器是以自回归形式输出，在处理LTSP任务中容易存在误差累积问题。针对该问题，提出了直接多步预测（DMS）策略，例如Informer设计了一种生成式的解码器用于直接多步（DMS）预测；Pyraformer使用一个连接时空轴的全连接层作为解码器；Autoformer分别从趋势周期分量和采用堆叠自相关机制（stacked auto-correlation）的季节分量来实现特征的精细分解，得到最终预测结果；FEDformer提出频率注意力块（frequency attention block）进行解码得到最终结果。</p>
<blockquote>
<p><strong>vanilla Transformer model</strong>：将原版的Transformer模型应用于LTSF任务，存在一定的局限性，包括自注意力机制存在的二次的时间/内存复杂度，自回归编码器存在的累积误差。</p>
<p><strong>Informer</strong>：针对上述问题所提出的用于降低复杂度的结构，并采用了直接多步（DMS）策略。</p>
<p><strong>autoformer</strong>：扩展改进了Informer模型，增加了自动关联机制，使得模型比标准注意力更好地学习时间依赖性，旨在准确分解时序数据的趋势和季节成分。特别的，该模型使用一个移动平均核来提取输入序列的趋势周期性（trend-cyclical）。</p>
<p><strong>FEDformer</strong>：进一步提出混合专家策略，设置多个大小的移动平均核来提取趋势分量。该模型侧重于在时间序列数据中捕捉全局特征，并提出了一个季节性趋势分解模型。</p>
<p><strong>Pyraformer</strong>：介绍了金字塔注意模型（PAM），其中尺度间树结构总结了不同分辨率下的特征，尺度内相邻连接对不同范围的时间依赖性进行建模。</p>
<p><strong>Earthformer</strong>：该模型专注于预测地球系统，如天气、气候和农业等。介绍了一种cuboid注意力架构。</p>
<p><strong>Non-Stationary Transformer</strong>：旨在调整Transformer以处理非平稳时间序列。提出了两种机制：去平稳注意力和一系列平稳化机制。这些机制在Informer、Autoformer、FEDformer和传统的Transformer等模型中都可以提高性能。</p>
</blockquote>
<p>论文认为现有的Transformer-based LTSF解决方案和non-Transformer模型进行性能对比时，所有的non-Transformer比较对象都是采用IMS预测技术（很容易受到累积误差的影响）。论文提出假设：现有的这些方案性能的提升主要是因为采用了DMS预测技术。为验证该假设，论文采用一种最简单的DMS模型LTSF-Linear作为基线模型，和现有的Transformer-based模型进行对比实验。LTSF-Linear是一系列线性模型的组合，不同变量之间共享权重，并且不对任何空间相关性（spatial correlations）进行建模。</p>
<p>实验在九个真实数据集上进行，结果表明LTSF-Linear性能优于现有的Transformer-based方法，并且还具有很大的优势。此外，通过消融实验探讨了LTSF模型各个组成部分对时间关系（temporal relation）提取的影响。</p>
<p>论文提出两种预处理方法的变体<strong>DLinear</strong>和<strong>NLinear</strong>，用于处理时序跨域问题（包括金融、交通和能源等领域）。DLinear可以增强原始线性模型对趋势明显的时序数据预测的性能。Linear可以提高LTSF-Linear模型对存在分布偏移（distribution shift）数据集处理能力。</p>
<p><strong>模型的评估方法</strong>：所有模型（除Earthformer）都在电力、交通、金融和天气数据集上进行了评估。使用的评价指标包括均方误差（MSE）和平均绝对误差（MAE）。</p>
<p>该论文只对比了Transformer-based模型，缺乏和其它简单模型的比较，如线性回归、LSTM/GRU、XGB等树形模型。改论文实验仅局限在一些标准数据集，在其他相关数据集上表现还有待验证，如Informer在预测河流流量存在巨大问题，与LSTM甚至是普通的Transformer相比，表现较差。</p>
<p>此外，时间序列数据在长度、周期性、趋势和季节性方面存在较大差异，因此模型需要更大范围的数据集来训练和验证。</p>
<p>所有的Transformer论文都同样存在有限评估问题，一个复杂的模型最初可能并不总是优于简单模型，这些需要在论文中进行严格比较和缺点的明确说明，而不是掩盖或者简单地假设没有这种情况。</p>
<blockquote>
<p>参考文献</p>
<p>Zeng A, Chen M, Zhang L, et al. Are transformers effective for time series forecasting?[C]//Proceedings of the AAAI conference on artificial intelligence. 2023, 37(9): 11121-11128.</p>
</blockquote>
<h2 id="术语理解"> 术语理解</h2>
<h3 id="自回归模型-autoregression-model"> 自回归模型（autoregression model）</h3>
<p>简称AR模型，统计上处理时序的方法，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.2806em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>t</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>与<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8917em;vertical-align:-0.2083em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>t</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span></span></span></span>相关，用<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8917em;vertical-align:-0.2083em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>t</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span></span></span></span>预测<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.2806em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>t</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>。一个最简单的预测为线性组合，即：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.2806em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>t</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span>ϕ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>0</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:2.9762em;vertical-align:-1.2777em;"></span><span><span><span><span style="height:1.6985em;"><span style="top:-1.8723em;margin-left:0em;"><span style="height:3.05em;"></span><span><span><span>i</span><span>=</span><span>1</span></span></span></span><span style="top:-3.05em;"><span style="height:3.05em;"></span><span><span>∑</span></span></span><span style="top:-4.3471em;margin-left:0em;"><span style="height:3.05em;"></span><span><span>p</span></span></span></span><span>​</span></span><span><span style="height:1.2777em;"><span></span></span></span></span></span><span style="margin-right:0.1667em;"></span><span><span>ϕ</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span style="margin-right:0.07847em;">X</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:-0.0785em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>t</span><span>−</span><span>i</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span>ϵ</span><span><span><span><span style="height:0.2806em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>t</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span></span></p>
<p>其中，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8889em;vertical-align:-0.1944em;"></span><span><span>ϕ</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>0</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>为常数项；<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span>ϵ</span><span><span><span><span style="height:0.2806em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>t</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>是零均值且方差为<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8141em;"></span><span><span style="margin-right:0.03588em;">σ</span><span><span><span><span style="height:0.8141em;"><span style="top:-3.063em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>2</span></span></span></span></span></span></span></span></span></span></span>的随机误差值（白噪声），其值对任何<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6151em;"></span><span>t</span></span></span></span>时刻都不变。</p>
<p>即：<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6833em;"></span><span style="margin-right:0.07847em;">X</span></span></span></span>的当前值等于一个或数个前期值的线性组合+常数项+随机误差。</p>
<h3 id="向量自回归-vector-autoregression"> 向量自回归（vector autoregression）</h3>
<p>描述在同一个样本期间内的<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>个变量（内生变量）可以作为它们过去值得线性函数。一个VAR模型可以写成：</p>
<p class='katex-block'><span><span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.2806em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>t</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span style="margin-right:0.2778em;"></span><span>=</span><span style="margin-right:0.2778em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>c</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8917em;vertical-align:-0.2083em;"></span><span><span>A</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>1</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>t</span><span>−</span><span>1</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.8917em;vertical-align:-0.2083em;"></span><span><span>A</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>2</span></span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.3011em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>t</span><span>−</span><span>2</span></span></span></span></span><span>​</span></span><span><span style="height:0.2083em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>⋯</span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.9694em;vertical-align:-0.2861em;"></span><span><span>A</span><span><span><span><span style="height:0.1514em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>p</span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span><span style="margin-right:0.03588em;">y</span><span><span><span><span style="height:0.2806em;"><span style="top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span><span>t</span><span>−</span><span>p</span></span></span></span></span><span>​</span></span><span><span style="height:0.2861em;"><span></span></span></span></span></span></span><span style="margin-right:0.2222em;"></span><span>+</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span>e</span><span><span><span><span style="height:0.2806em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>t</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span></span></p>
<p>其中，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.4306em;"></span><span>c</span></span></span></span>是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>n</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>1</span></span></span></span>得常数向量，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.8333em;vertical-align:-0.15em;"></span><span><span>A</span><span><span><span><span style="height:0.3117em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>i</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>n</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.4306em;"></span><span>n</span></span></span></span>矩阵，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.5806em;vertical-align:-0.15em;"></span><span><span>e</span><span><span><span><span style="height:0.2806em;"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em;"><span style="height:2.7em;"></span><span><span>t</span></span></span></span><span>​</span></span><span><span style="height:0.15em;"><span></span></span></span></span></span></span></span></span></span>是<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.6667em;vertical-align:-0.0833em;"></span><span>n</span><span style="margin-right:0.2222em;"></span><span>×</span><span style="margin-right:0.2222em;"></span></span><span><span style="height:0.6444em;"></span><span>1</span></span></span></span>误差向量，<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>p</span></span></span></span>是最多使用<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span style="margin-right:0.03588em;">y</span></span></span></span>的<span><span><i>Not supported content</i></span><span aria-hidden="true"><span><span style="height:0.625em;vertical-align:-0.1944em;"></span><span>p</span></span></span></span>个滞后期。</p>
<p>VAR模型是一种用于多变量时间序列分析的统计模型，特别是在变量之间有相互影响关系的时间序列。VAR模型的方程随着时间序列中变量数量的增加而增加，因此它允许对多变量时序数据进行分析和预测，被广泛应用于经济学和天气预报中。</p>
<p>ARIMA系列与VAR模型的基本区别在于，所有ARIMA模型都用于单变量时间序列，而VAR模型则用于多变量时间序列。此外，ARIMA模型是单向模型，这意味着因变量受其过去或滞后值本身的影响，而VAR是双向模型，这意味着因变量受其过去的值或另一变量的值的影响，或受两者的影响。</p>
<p>链接：https://juejin.cn/post/7087845757713121310</p>
<h3 id="归纳偏置-inductive-bias"> 归纳偏置（inductive bias）</h3>
<p>机器学习中很多算法会对要处理的任务做一些关于目标函数的必要假设，称为归纳偏置。【<strong>归纳性偏好</strong>】</p>
<p>通俗理解：归纳偏置是从现实中的一些现象归纳出一定的规则（heuristic），然后对模型做一定的约束，从而起到“模型选择”的作用，类似贝叶斯学习中的“先验”。</p>
<p>西瓜书解释：机器学习算法在学习过程中对某种类型假设的 偏好，称为 归纳偏好。归纳偏好可以看作学习算法自身在一个庞大的假设空间中对假设进行 选择 的 启发式 或 “价值观”。</p>
<p>维基百科解释：如果学习器需要去预测 “其未遇到过的输入” 的结果时，则需要一些 假设 来 帮助它做出选择。</p>
<p>广义解释：归纳偏置会促使学习算法优先考虑具有某些属性的解。</p>
<p>例如：深度神经网络偏好性地认为，层次化处理信息有更好效果；循环神经网络认为信息具有空间局限性，可用滑动卷积共享权重的方式降低参数空间；图网络认为中心节点与邻居节点的相似性会更好地引导信息流动。</p>
<p>通常，模型容量（capacity）很大但inductive bias匮乏，容易导致过拟合（overfitting），如Transformer。</p>
<blockquote>
<p>引用自：https://blog.csdn.net/qq_39478403/article/details/121107057</p>
</blockquote>
<p>网络架构搜索（neural architecture search）</p>
<p>机器学习算法的效果很大程度上取决于各种超参数，深度学习兴起之前，超参数自动搜索优化方法主要包括：随机搜索（random search）、网络搜索（grid search）、贝叶斯优化（Bayesian optimization）、强化学习（reinforcement learning）、进化算法（evolutional algorithm）等，统称为Hyperparameter optimization（HO）。</p>
<p>深度学习领域中主要涉及到的超参数分为两类：（1）训练参数，包括learning rate、batch-size，weight decay等；（2）网络结构参数，包括网络层数，每一层算子，卷积中的filter size等。对于（1）中的参数自动调优仍属于HO范畴，（2）中参数具有纬度高，离散且相互依赖等特点，其自动调优一般称为网络结构搜索（NAS）。</p>
<p>网络结构的设计很大程度上还是需要hand-craft，且依赖经验。寻求各种新的更优的网络子结构是一个研究热点。</p>
<blockquote>
<p>参考资料：https://blog.csdn.net/jinzhuojun/article/details/84698471</p>
</blockquote>
]]></content:encoded>
    </item>
    <item>
      <title>DL for Load Forecasting sequence to sequence RNNs with Attention</title>
      <link>https://jiangxj.top/vuepress/vuepress/note/%E6%97%B6%E5%BA%8F%E9%A2%84%E6%B5%8B%E8%AE%BA%E6%96%87/DL%20for%20Load%20Forecasting%20sequence%20to%20sequence%20RNNs%20with%20Attention/</link>
      <guid>https://jiangxj.top/vuepress/vuepress/note/%E6%97%B6%E5%BA%8F%E9%A2%84%E6%B5%8B%E8%AE%BA%E6%96%87/DL%20for%20Load%20Forecasting%20sequence%20to%20sequence%20RNNs%20with%20Attention/</guid>
      <source url="https://jiangxj.top/vuepress/vuepress/rss.xml">DL for Load Forecasting sequence to sequence RNNs with Attention</source>
      <description>注意力机制结合Seq2Seq用于负载预测</description>
      <category>PapersReading</category>
      <pubDate>Fri, 20 Oct 2023 00:00:00 GMT</pubDate>
      <content:encoded><![CDATA[<blockquote>
<p>L. Sehovac and K. Grolinger, &quot;Deep Learning for Load Forecasting: Sequence to Sequence Recurrent Neural Networks With Attention,&quot; in IEEE Access, vol. 8, pp. 36411-36426, 2020, doi: 10.1109/ACCESS.2020.2975738.</p>
</blockquote>
<h2 id="abstract"> Abstract</h2>
<p><strong>Electrical load forecasting</strong> is essential as it supports energy production planning and scheduling, assists with budgeting, and helps identify saving opportunities.
This paper proposes <strong>Sequence to Sequence Recurrent Neural Network (S2S RNN) with Attention</strong> for electrical load forecasting.
The S2S architecture from language translation is adapted for load forecasting and a corresponding sample generation approach is designed.
RNN enables capturing time dependencies present in the load data and S2S model further improves time modeling by combining two RNNs: encoder and decoder.
The Attention mechanism alleviates the burden of connecting encoder and decoder.</p>
<h2 id="introduction"> Introduction</h2>
<p>Load forecasting has been attracting research and industry attention because of its importance for energy production planning and scheduling.
The rapid increase of smart meter(智慧仪表) use has created opportunities for load forecasting.
Energy consumption data from smart meters is used to build models capable of inferring future energy consumption, thus facilitating cost-effective decision-making.</p>
<h2 id="methods-for-load-forecasting"> Methods for load forecasting</h2>
<p>Load forecasting can be classified into three main categories: short, medium, and long-term.
There are many approaches to load forecasting, including physics, statistics, and machine learning-based. This paper focuses on machine learning-based models.</p>
<h2 id="ml-models-for-load-forecasting"> ML models for load forecasting</h2>
<ul>
<li>Feedfoward Neural Networks(FFNNs)</li>
<li>Support Vector Regression(SVR)</li>
<li>Recurrent Neural Networks(RNNs)</li>
</ul>
<p>RNN have an advantage over DNNs in analyzing the temporal dynamic behavior, in language translation a <strong>Sequence-to-Sequence(S2S) RNN</strong> which combines an encoder RNN and decoder RNN has shown a greater success.</p>
<h3 id="s2s-rnn"> S2S RNN</h3>
<p>RNNs  are good at modeling temporal behaviors found in time series data. RNNs take a sequence of inputs <img src="https://cdn.nlark.com/yuque/__latex/fdf8c4b58388f9b1ddf57b8b15cc6410.svg#card=math&amp;code=x_{[1]}%2C\dots%2Cx_{[T]}&amp;id=Ko0lk" alt="" loading="lazy">, and previous hidden states, to compute a sequence of outputs <img src="https://cdn.nlark.com/yuque/__latex/b93770825595e0a3af24b3afbf1f361c.svg#card=math&amp;code=y_{[1]}%2C\dots%2Cy_{[T]}&amp;id=o9upy" alt="" loading="lazy">.
<strong>Vanilla RNNs</strong> are mainly trained using back-propagation through time(BPTT), which can lead to the vanishing gradient problem for long sequences. Long-short-term memory(LSTM) networks  were designed to overcome this problem.
**LSTM **networks are capable of storing information for longer periods of time and the model can make better predicitons.
The <strong>Gated Recurrent Unit(GRU)</strong> was introduced to simplify the LSTM model. As GRUs have fewer parameters, convergence is achieved faster than with LSTMs.
<strong>Sequence to Sequence (S2S or Seq2seq) RNNs</strong> consist of an encoder and decoder RNN.</p>
<h3 id="attention-mechanism-for-s2s"> Attention Mechanism for S2S</h3>
<p>In S2S model, the encoder is burdened with compressing all necessary information into a fixed-length vector. It was addressed by attention mechanism in language translation.
Attention mechanism allows the decoder to look back at the encoder outputs to find the most relevant information.</p>
<h2 id="methodology"> Methodology</h2>
<h3 id="pre-processing"> Pre-processing</h3>
<p>Dataset contains nine futures: months, day of year, day of month, weekday, weekend, holiday, hour, season, and energy usage. The dataset was divided into a training set(80%) and a test set(20%)。
The value of each future in the data were transformed to have zero-mean and unit-variance.</p>
<h3 id="sample-generation"> Sample generation</h3>
<h3 id="s2s-prediction-with-bahdanau-attention"> S2S Prediction with Bahdanau Attention</h3>
<h3 id="s2s-prediction-with-luong-attention"> S2S Prediction with Luong Attention</h3>
<h2 id="experiments"> Experiments</h2>
<p>1、The proposed approach was evaluated on a real-world dataset from a commercial building provided by an industry partner. The dataset contained one year and three months of energy load data in five minute intervals.</p>
<p>2、All S2S models were tested for four different prediction lengths <img src="https://cdn.nlark.com/yuque/__latex/6cb78cc5fb5f9fbb03c8a72c2a470201.svg#card=math&amp;code=N%3D[12%2C48%2C120%2C288]&amp;id=zU9wX" alt="" loading="lazy">.</p>
<p>3、Four inputs cases with different input lengths<img src="https://cdn.nlark.com/yuque/__latex/40b1d3b0db269d160b805c7957e8a83a.svg#card=math&amp;code=T%3D[12%2C48%2C120%2C288]&amp;id=ZhTEV" alt="" loading="lazy">were chosen for experiments.</p>
<p>4、The four input cases with four prediction lengths, makes a total of 16 cases considered.</p>
<p>5、With each of the 16 cases, 24 different models were evaluated:</p>
<ul>
<li>S2S-o model with GRU/LSTM/RNN cells (3 models)</li>
<li>S2S-BA model with GRU/LSTM/RNN cells (3 models)</li>
<li>S2S-LA model with GRU/LSTM/RNN cells, accompanied with each cell is one of three attention score function: dot, general, concat (9 models)</li>
<li>Non-S2S RNN, one layer with GRU/LSTM/RNN cell (3 models)</li>
<li>Non-S2S RNN, three layers with GRU/LSTM/RNN cell (3 models)</li>
<li>DNN model with sizes: small, medium, and large (3 models)</li>
</ul>
<p>6、The accuracy measures used throughout this work are the Mean Absolute Error(<strong>MAE</strong>) and Mean Absolute Percentage Error(<strong>MAPE</strong>).</p>
<h2 id="results-and-discussion"> Results and Discussion</h2>
<p>1、The prediction length <img src="https://cdn.nlark.com/yuque/__latex/459f3c80a50b7be28751b0869ef5386a.svg#card=math&amp;code=N&amp;id=LJvzI" alt="" loading="lazy">increases, the accuracy decreases.
2、S2S-BA model was the dominant(占优势的) model.
3、The one layer Non-S2S outperformed the three layer Non-S2S model what demonstrated that additional layers did not improve the forecasting accuracy.
4、All S2S models performed better than one layer Non-S2S models.
5、The attention models contain more paremeters than other models.
6、The S2S-o model achives comparable results to all attention-based models for each prediction length while being faster to train because of having fewer parameters.
7、The S2S-BA model is prefered for high accuracy irrelevant of training speed.
8、The S2S-o model is prefered for slight decrease in accuracy and reduction in training speed.
9、The S2S-BA model with GRU is not showing signs of overfitting. While NNs with a large number of weights such as S2S with attention are prone to overfitting.</p>
]]></content:encoded>
      <enclosure url="https://cdn.nlark.com/yuque/__latex/fdf8c4b58388f9b1ddf57b8b15cc6410.svg#card=math&code=x_%7B%5B1%5D%7D%2C%5Cdots%2Cx_%7B%5BT%5D%7D&id=Ko0lk" type="image/"/>
    </item>
  </channel>
</rss>