筆記：Few-shot learning for tackling open-set generalization：

• 基于點(diǎn)云的語義分割的應(yīng)用：場景理解，給點(diǎn)云中每一個(gè)點(diǎn)賦予特點(diǎn)的語義標(biāo)簽。（如自動(dòng)駕駛）

• 小樣本學(xué)習(xí)的意義：解決太過于依賴大量標(biāo)定數(shù)據(jù)，減少成本；可以提高泛化能力,識(shí)別未曾見過的目標(biāo)。

• paper1:Few-shot 3D Point Cloud Semantic Segmentation

  • 提出問題：

    • rely on large amounts of labeled training data, so they are time-consuming and expensive to collect.
    • follow the closed set assumption.(訓(xùn)練集和測試集取自同一label space) ，泛化能力差。

  • 解決：

    • multi-prototype transductive inference method.

      • transductive inference: 轉(zhuǎn)導(dǎo)推理；是一種通過觀察特點(diǎn)的樣本，進(jìn)而預(yù)測特定的測試樣本的方法，是一種從特殊到特殊的推理，適合于小樣本推理。不同于歸納推理，先從訓(xùn)練樣本中學(xué)習(xí)規(guī)則，再用規(guī)則判斷測試樣本。

    • architecture：
      

      • embedding network:

        • three properties:1.local geometric features; 2.global geometric features; 3. adapt to different few-shot tasks.
        • DGGNN: the backbone of feature extractor.(local)
        • SAN(self-attention network): generate semantic feature.(global)
        • MLP: adapt to different few-shot tasks.

      • multi-prototype generation:

        • It samples a subset of n seed points from a set of support points in one class using the farthest point sampling based on the embedding space.(對support set的每一類樣本點(diǎn)farthest points sample，抽取n個(gè)seed point)
        • The farthest points represent different perspectives of one class. (farthest points sample保證足夠的感受野)

      • transductive inference:

        • use transductive label propagation to construct a graph on the labeled multi-prototypes and the unlabeled query points.(用k-NN建立相關(guān)類的圖)

      • label propagation

      • cross-entropy loss function(交叉熵?fù)p失函數(shù)):

        • compute the cross-entropy loss with ground truth labels.

• paper2:What Makes for Effective Few-shot Point Cloud Classification?

  • 提出問題：

    • they require extensive data collection and retraining when dealing with novel classes never seen before.
    • It is hard to study from existing 2D methods when migrating to the 3D domain.
    • point clouds are more complex and have unorder structure in European space.

  • 3D point cloud classification

    • projection-based: It first converts the irregular points into a representation like voxel, pillar, and then apply typical 2D or 3D CNN to extract features.
    • point-based: It can learns point-wise features with multilayer perceptron(MLP) and aggregates global feature with a symmetric function implemented by a max-pooling layer.

  • 2D few-shot learning

    • Metric-based: It focus on learning an embedding space where similar samples pairs are closer, or designing a metric function to compare the feature similarity of samples.
    • Optimization-based: It regards meta-learning as an optimization process.

  • State-of-the-art 2D FSL on Point Cloud

    • compare the metric-based methods and optimization-based methods, and concludes that metric-based methods outperform the optimization-based methods in point cloud scenario.

  • Influence of Backbone Architecture on FSL

    • select three types of current state-of-the-art 3D point-based networks including Pointwise-based， Convolution-based， Graph-based(DGCNN). One can conclude that the graph-based network DGCNN achieves higher classification accuracy than other networks on these two datasets.

  • Cross Instance Adaption (CIA) module

    • CIA can be inserted into existing backbones and learning frameworks to learn more discriminative representations for the support set and query set.

      Embedding module把support-set和query-set作為輸入分別進(jìn)行特征提取得到他們的prototype，然后再通過CIA模塊更新support-set和query-set，然后在特征空間計(jì)算每個(gè)class prototype和query examples的歐氏距離，最后便可得到損失函數(shù)并進(jìn)行優(yōu)化。

    • Self-Channel Interaction Module: address the issues of subtle inter-class differences.

      • 先從embedding space分別由兩個(gè)線性系數(shù)φ和γ得到q向量和k向量，然后通過CIM的雙線性變換得到一個(gè)channel-wise relation score map - R, 然后進(jìn)行softmax操作得到權(quán)重矩陣R’，最后得到更新的向量v是有R’與開始的特征向量加權(quán)和得到，vi越大說明特供信息越大，有利于區(qū)分class之間的細(xì)小差別。

    • Cross-Instance Fusion Module: address high intra-class variances issues

    • 首先將support feature和query feature 連結(jié)起來得到Z,然后用兩個(gè)卷積層來解碼連結(jié)后的特征得到W，將W進(jìn)行softmax操作得到權(quán)值矩陣后與Z點(diǎn)乘來更新support feature和query feature。

  • 本文還提供了兩個(gè)適用于3D FSL的數(shù)據(jù)集：ModelNet40-FS，ShapeNet70-FS

總結(jié)

以上是生活随笔為你收集整理的笔记：基于点云的语义分割的小样本学习的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。