轉(zhuǎn)自AI Studio，原文鏈接：

LIC 2022 視頻語義理解基線（快速啟動(dòng)版） - 飛槳AI Studio

LIC2022視頻語義理解基線

??該版本為快速啟動(dòng)版，訓(xùn)練集取比賽提供的訓(xùn)練集的子集，目的是減少在線項(xiàng)目加載等待時(shí)間，供選手快速跑通流程，實(shí)現(xiàn)一鍵訓(xùn)練+預(yù)測，生成可提交并有分?jǐn)?shù)的結(jié)果文件。

該基線分?jǐn)?shù)比全量數(shù)據(jù)版基線（🚩鏈接稍后給出）稍低（e.g., 0.33 vs 0.38）。

??請(qǐng)選擇啟動(dòng)GPU環(huán)境（CPU環(huán)境將無法進(jìn)行模型的訓(xùn)練和預(yù)測，且從CPU到GPU環(huán)境的轉(zhuǎn)換將非常慢）。

??每次打開環(huán)境，需要約15分鐘文件的加載、同步時(shí)間，請(qǐng)耐心等候加載完畢。

😃 每天AI Studio 免費(fèi)提供8個(gè)算力點(diǎn)，最多可使用16小時(shí)GPU。除此之外，選手還可以選擇申請(qǐng)AI Studio提供的100小時(shí)免費(fèi)V100 GPU算力碼。

1. 背景介紹

該項(xiàng)目為👉?LIC2022視頻語義理解技術(shù)評(píng)測任務(wù)👈 的快速啟動(dòng)版基準(zhǔn)模型，項(xiàng)目基于github基線改造，為選手提供了存儲(chǔ)和算力支持。模型包括兩部分：1）視頻分類標(biāo)簽?zāi)Ｐ?paddle-video-classify-tag；2）視頻語義標(biāo)簽?zāi)Ｐ?paddle-video-semantic-tag。

視頻分類標(biāo)簽?zāi)Ｐ透鶕?jù)視頻內(nèi)容在封閉的二級(jí)標(biāo)簽體系上進(jìn)行分類，得到描述視頻的分類標(biāo)簽；視頻語義標(biāo)簽?zāi)Ｐ蛷囊曨l的文本信息中抽取實(shí)體語義標(biāo)簽（選手可進(jìn)行升級(jí)，如利用給定的知識(shí)庫進(jìn)行推理、融合多模信息提升標(biāo)簽理解效果，或生成標(biāo)簽等）。兩部分模型產(chǎn)出的標(biāo)簽結(jié)果，分別對(duì)應(yīng)技術(shù)評(píng)測標(biāo)簽中的分類標(biāo)簽、語義標(biāo)簽。

2. 快速實(shí)踐

2.1 環(huán)境搭建

運(yùn)行下列指令添加路徑。

注：注釋行中的依賴包已持久化安裝至項(xiàng)目目錄，無需重復(fù)安裝。

In [?]

# !mkdir /home/aistudio/external-libraries # !pip install opencv-python -i https://mirror.baidu.com/pypi/simple -t /home/aistudio/external-libraries # !pip install paddlenlp==2.0.1 -i https://mirror.baidu.com/pypi/simple -t /home/aistudio/external-libraries # !pip install tqdm wget -t /home/aistudio/external-libraries !tar -xvf external-libraries.tar !rm external-libraries.tar import os import sys sys.path.append('/home/aistudio/external-libraries')!mkdir paddle-video-classify-tag && cd paddle-video-classify-tag && mkdir data !mkdir paddle-video-semantic-tag && cd paddle-video-semantic-tag && mkdir data

2.2 數(shù)據(jù)加載

數(shù)據(jù)內(nèi)容包含：

• 樣例訓(xùn)練集（比賽使用訓(xùn)練集的抽樣集合）的視頻信息，及官方提供的tsn視覺特征

• A榜測試集（比賽使用的A榜測試集全量集合）的視頻信息，及官方提供的tsn視覺特征

In [?]

!tar -zxvf /home/aistudio/data/data142559/dataset_sample.tar.gz !cd paddle-video-classify-tag && unzip /home/aistudio/data/data142559/tsn_features_test_a.zip !cd paddle-video-classify-tag && unzip /home/aistudio/data/data142559/tsn_features_train_sample.zip

2.3 視頻分類標(biāo)簽基線

該基線基于?VideoTag飛槳大規(guī)模視頻分類模型?，能夠根據(jù)視頻內(nèi)容在封閉的二級(jí)標(biāo)簽體系上進(jìn)行分類，得到描述視頻的分類標(biāo)簽。

2.3.1 數(shù)據(jù)準(zhǔn)備

該視頻分類標(biāo)簽?zāi)Ｐ褪褂肨SN網(wǎng)絡(luò)提取原始視頻的向量表征。 由于該步驟比較耗時(shí)，我們提供了所有視頻數(shù)據(jù)的TSN特征，運(yùn)行2.2數(shù)據(jù)加載 章節(jié)指令即可。

數(shù)據(jù)集上有兩級(jí)標(biāo)簽，我們分別在一級(jí)標(biāo)簽（level1）和二級(jí)標(biāo)簽（level2）的設(shè)定下進(jìn)行分類實(shí)驗(yàn)。 在每一種設(shè)定下均需要進(jìn)行訓(xùn)練、驗(yàn)證和測試的數(shù)據(jù)劃分。準(zhǔn)備過程會(huì)得到如下的輸出：

paddle-video-classify-tag|-- weights|-- attention_lstm.pdmodel|-- attention_lstm.pdopt|-- attention_lstm.pdparams

運(yùn)行以下代碼，準(zhǔn)備視頻語義理解數(shù)據(jù)集的label集合；訓(xùn)練、驗(yàn)證、測試的樣本列表等。

In [?]

import os import os.path as osp import requests import time import codecs import json import argparse import randomdef create_splits_indice(n_samples, SPLITS):assert sum([v for k, v in SPLITS]) == 1.0indices = list(range(n_samples))random.shuffle(indices)split2indice = {}r_offset = 0for idx, (split, ratio) in enumerate(SPLITS):l_offset = r_offsetif idx == len(SPLITS) - 1:r_offset = n_sampleselse:r_offset = int(n_samples * ratio) + l_offsetsplit2indice[split] = indices[l_offset:r_offset]return split2indicedef prepare_split(data, split_name, test_only=False, gather_labels=False, classify_tag_dir='/home/aistudio/paddle-video-classify-tag'):'''1. Prepare ALL (unique) labels for classification from trainval-set.2. For each split, generate sample list for level1 & level2 classification.'''trainval_tsn_feature_dir = '/home/aistudio/paddle-video-classify-tag/tsn_features_train_sample'test_tsn_feature_dir = '/home/aistudio/paddle-video-classify-tag/tsn_features_test_a'cls_data_dir = os.path.join(classify_tag_dir, 'data')if not os.path.exists(cls_data_dir):os.mkdir(cls_data_dir)sample_nids = [sample["@id"] for sample in data]level1_labels = []level2_labels = []if not test_only:for sample in data:category = {each["@meta"]["type"]: each["@value"]for each in sample["category"]}level1_labels.append(category["level1"])level2_labels.append(category["level2"])def create_sample_list(sample_labels, level_name):save_label_file = cls_data_dir + "/{}_label.txt".format(level_name)if gather_labels:# For trainval set:# Gather candidate labels and dump to {level1,level2}_label.txtlabels = sorted([str(label) for label in list(set(sample_labels))])with codecs.open(save_label_file, "w", encoding="utf-8") as ouf:ouf.writelines([label + "\n" for label in labels])print("Saved " + save_label_file)else:# For test set: load existing labels.with codecs.open(save_label_file, "r", encoding="utf-8") as inf:labels = [line.strip() for line in inf.readlines()]label2idx = {label: idx for idx, label in enumerate(labels)}sample_lines = []# Generate sample list: one sample per line (feature_path -> label)for i in range(len(sample_nids)):label_indice = label2idx[str(sample_labels[i])] if not test_only \else -1if split_name in ["train", "val", "trainval"]:tsn_feature_dir = trainval_tsn_feature_direlif split_name in ["test"]:tsn_feature_dir = test_tsn_feature_dirfeature_path = osp.join(tsn_feature_dir,"{}.npy".format(sample_nids[i]))if osp.exists(feature_path):line = "{} {}\n".format(feature_path, str(label_indice))sample_lines.append(line)save_split_file = cls_data_dir + "/{}_{}.list".format(level_name, split_name)with codecs.open(save_split_file, "w", encoding="utf-8") as ouf:ouf.writelines(sample_lines)print("Saved {}, size={}".format(save_split_file,len(sample_lines)))create_sample_list(level1_labels, "level1")create_sample_list(level2_labels, "level2")random.seed(6666) classify_tag_dir = '/home/aistudio/paddle-video-classify-tag' if not os.path.exists(classify_tag_dir):os.mkdir(classify_tag_dir) trainval_path = '/home/aistudio/dataset_sample/train.sample.json' test_path = '/home/aistudio/dataset_sample/test_a.json'# load data for train & validation (have labels). with codecs.open(trainval_path, "r", encoding="utf-8") as inf:print("Loading {}...".format(trainval_path))lines = inf.readlines()trainval_data = [json.loads(line) for line in lines]# load data for test (no labels). with codecs.open(test_path, "r", encoding="utf-8") as inf:print("Loading {}...".format(test_path))lines = inf.readlines()test_data = [json.loads(line) for line in lines]# split the trainval data into train-set(80%) and validation-set(20%). split2indice = create_splits_indice(len(trainval_data), [("train", 4.0 / 5.0),("val", 1.0 / 5.0),]) train_data = [trainval_data[idx] for idx in split2indice["train"]] val_data = [trainval_data[idx] for idx in split2indice["val"]]prepare_split(trainval_data, "trainval", gather_labels=True) prepare_split(train_data, "train") prepare_split(val_data, "val") prepare_split(test_data, "test", test_only=True) Loading /home/aistudio/dataset_sample/train.sample.json... Loading /home/aistudio/dataset_sample/test_a.json...

2.3.2 訓(xùn)練與驗(yàn)證

運(yùn)行以下指令訓(xùn)練分類模型。

選手可以參考原代碼庫中的模型微調(diào)指南獲取更多信息。

In [4]

import os import sys import argparse import ast import logging import paddle paddle.enable_static() sys.path.append('/home/aistudio/external-libraries') sys.path.append('/home/aistudio/work/paddle-video-classify-tag')from utils.train_utils import train_with_dataloader import models from utils.config_utils import * from reader import get_reader from metrics import get_metrics from utils.utility import check_cuda from utils.utility import check_versionlogging.root.handlers = [] FORMAT = '[%(levelname)s: %(filename)s: %(lineno)4d]: %(message)s' logging.basicConfig(level=logging.INFO, format=FORMAT) logger = logging.getLogger(__name__)class Args():model_name = 'AttentionLSTM'config = '/home/aistudio/work/paddle-video-classify-tag/configs/attention_lstm-single-level1.yaml'batch_size = Nonelearning_rate = Nonepretrain = '/home/aistudio/work/paddle-video-classify-tag/weights/attention_lstm'use_gpu = Trueno_memory_optimize = Falseepoch = Nonevalid_interval = 1save_dir = os.path.join('paddle-video-classify-tag', 'data', 'checkpoints', 'level1')log_interval = 50fix_random_seed = Falsedef train(args):# parse configconfig = parse_config(args.config)train_config = merge_configs(config, 'train', vars(args))valid_config = merge_configs(config, 'valid', vars(args))# print_configs(train_config, 'Train')train_model = models.get_model(args.model_name, train_config, mode='train')valid_model = models.get_model(args.model_name, valid_config, mode='valid')# build modelstartup = paddle.static.Program()train_prog = paddle.static.Program()if args.fix_random_seed:startup.random_seed = 1000train_prog.random_seed = 1000with paddle.static.program_guard(train_prog, startup):with paddle.utils.unique_name.guard():train_model.build_input(use_dataloader=True)train_model.build_model()# for the input, has the form [data1, data2,..., label], so train_feeds[-1] is labeltrain_feeds = train_model.feeds()train_fetch_list = train_model.fetches()train_loss = train_fetch_list[0]optimizer = train_model.optimizer()optimizer.minimize(train_loss)train_dataloader = train_model.dataloader()valid_prog = paddle.static.Program()with paddle.static.program_guard(valid_prog, startup):with paddle.utils.unique_name.guard():valid_model.build_input(use_dataloader=True)valid_model.build_model()valid_feeds = valid_model.feeds()valid_fetch_list = valid_model.fetches()valid_dataloader = valid_model.dataloader()place = paddle.CUDAPlace(0) if args.use_gpu else paddle.CPUPlace()exe = paddle.static.Executor(place)exe.run(startup)if args.pretrain:train_model.load_pretrain_params(exe, args.pretrain, train_prog)build_strategy = paddle.fluid.compiler.BuildStrategy()build_strategy.enable_inplace = Trueexec_strategy = paddle.static.ExecutionStrategy()compiled_train_prog = paddle.static.CompiledProgram(train_prog).with_data_parallel(loss_name=train_loss.name,build_strategy=build_strategy,exec_strategy=exec_strategy)compiled_valid_prog = paddle.static.CompiledProgram(valid_prog).with_data_parallel(share_vars_from=compiled_train_prog,build_strategy=build_strategy,exec_strategy=exec_strategy)# get readerbs_denominator = 1if args.use_gpu:# check number of GPUsgpus = os.getenv("CUDA_VISIBLE_DEVICES", "")if gpus == "":passelse:gpus = gpus.split(",")num_gpus = len(gpus)assert num_gpus == train_config.TRAIN.num_gpus, \"num_gpus({}) set by CUDA_VISIBLE_DEVICES " \"shoud be the same as that " \"set in {}({})".format(num_gpus, args.config, train_config.TRAIN.num_gpus)bs_denominator = train_config.TRAIN.num_gpustrain_config.TRAIN.batch_size = int(train_config.TRAIN.batch_size /bs_denominator)valid_config.VALID.batch_size = int(valid_config.VALID.batch_size /bs_denominator)train_reader = get_reader(args.model_name.upper(), 'train', train_config)valid_reader = get_reader(args.model_name.upper(), 'valid', valid_config)# get metricstrain_metrics = get_metrics(args.model_name.upper(), 'train', train_config)valid_metrics = get_metrics(args.model_name.upper(), 'valid', valid_config)epochs = args.epoch or train_model.epoch_num()exe_places = paddle.static.cuda_places() if args.use_gpu else paddle.static.cpu_places()train_dataloader.set_sample_list_generator(train_reader, places=exe_places)valid_dataloader.set_sample_list_generator(valid_reader, places=exe_places)train_with_dataloader(exe,train_prog,compiled_train_prog,train_dataloader,train_fetch_list,train_metrics,epochs=epochs,log_interval=args.log_interval,valid_interval=args.valid_interval,save_dir=args.save_dir,save_model_name=args.model_name,fix_random_seed=args.fix_random_seed,compiled_test_prog=compiled_valid_prog,test_dataloader=valid_dataloader,test_fetch_list=valid_fetch_list,test_metrics=valid_metrics)args = Args()# check whether the installed paddle is compiled with GPU check_cuda(args.use_gpu) check_version()args.model_name = 'AttentionLSTM' args.log_interval = 50# first layer args.config = '/home/aistudio/work/paddle-video-classify-tag/configs/attention_lstm-single-level1.yaml' args.save_dir = '/home/aistudio/paddle-video-classify-tag/data/checkpoints/level1' if not os.path.exists(args.save_dir):os.makedirs(args.save_dir) train(args)# second layer args.config = '/home/aistudio/work/paddle-video-classify-tag/configs/attention_lstm-single-level2.yaml' args.save_dir = '/home/aistudio/paddle-video-classify-tag/data/checkpoints/level2' if not os.path.exists(args.save_dir):os.makedirs(args.save_dir) train(args) [INFO: regularizer.py: 101]: If regularizer of a Parameter has been set by 'fluid.ParamAttr' or 'fluid.WeightNormParamAttr' already. The Regularization[L2Decay, regularization_coeff=0.000800] in Optimizer will not take effect, and it will only be applied to other Parameters! [INFO: attention_lstm.py: 164]: Load pretrain weights from /home/aistudio/work/paddle-video-classify-tag/weights/attention_lstm, exclude fc layer. [INFO: train_utils.py: 45]: ------- learning rate [0.000125], learning rate counter [-] ----- [INFO: metrics_util.py: 80]: [TRAIN 2022-04-28 22:05:55] Epoch 0, iter 0, time 2.767573356628418, , loss = 2923.538330, Hit@1 = 0.02, PERR = 0.02, GAP = 0.02 [INFO: train_utils.py: 122]: [TRAIN] Epoch 0 training finished, average time: 1.7017907901686065 share_vars_from is set, scope is ignored. [INFO: metrics_util.py: 80]: [TEST] test_iter 0 , loss = 302.009247, Hit@1 = 0.53, PERR = 0.53, GAP = 0.60 [INFO: metrics_util.py: 124]: [TEST] Epoch0 Finish avg_hit_at_one: 0.505859375, avg_perr: 0.505859375, avg_loss :329.6025136311849, aps: [0.2056517240004745, 0.7582162120652176, 0.06977232338023834, 0, 0.0, 0.8453953428088683, 0.5386290618179087, 0.12647074684748238, 0.06464234760008425, 0.8143473229661798, 0.025846861750262674, 0.8125, 0.5606631794981926, 0.025, 0.6332121597279986, 0.48199568449775865, 0.03583711373823246, 0.844207523649567, 0.006626716822675181, 0.8250196464463089, 0.5152466244119106, 0.27196454054549163, 0.6783860174137636, 0, 0.053316102627013265, 0.19644972430583615, 0.9643802486046634, 0.17060293754538206, 0.05704956089391827, 0.07652161831232097, 0, 0.038493472255469084, 0.5543688830575145], gap:0.5390077300563219 [INFO: train_utils.py: 45]: ------- learning rate [0.000125], learning rate counter [-] ----- [INFO: metrics_util.py: 80]: [TRAIN 2022-04-28 22:08:10] Epoch 1, iter 0, time 2.6242427825927734, , loss = 364.424683, Hit@1 = 0.48, PERR = 0.48, GAP = 0.44 [INFO: train_utils.py: 122]: [TRAIN] Epoch 1 training finished, average time: 1.7430822508675712 [INFO: metrics_util.py: 80]: [TEST] test_iter 0 , loss = 248.946411, Hit@1 = 0.62, PERR = 0.62, GAP = 0.70 [INFO: metrics_util.py: 124]: [TEST] Epoch1 Finish avg_hit_at_one: 0.5944010416666666, avg_perr: 0.5944010416666666, avg_loss :282.882386525472, aps: [0.2639931946892855, 0.85903455775213, 0.09778176122448987, 0, 0.030033261592643315, 0.8680903206216465, 0.6081577153299009, 0.22149620838665163, 0.09123414426924248, 0.8905820604115383, 0.055248944301131656, 0.95, 0.5856899952942017, 0.168086815154962, 0.6645151962662453, 0.5090036826258103, 0.17872309016688617, 0.8700421734225023, 0.03301833984693047, 0.8629644432673815, 0.5494279056102854, 0.33194993658077027, 0.7208050878611371, 0, 0.2092875916594547, 0.24870436262552853, 0.9722225403199237, 0.19206833214739047, 0.09036579340064066, 0.10623058817084435, 0, 0.03769712329789659, 0.6627263444618232], gap:0.6346135299340665 [INFO: train_utils.py: 45]: ------- learning rate [0.000125], learning rate counter [-] ----- [INFO: metrics_util.py: 80]: [TRAIN 2022-04-28 22:10:24] Epoch 2, iter 0, time 1.708108901977539, , loss = 315.439453, Hit@1 = 0.55, PERR = 0.55, GAP = 0.56 [INFO: train_utils.py: 122]: [TRAIN] Epoch 2 training finished, average time: 1.795787981578282 [INFO: metrics_util.py: 80]: [TEST] test_iter 0 , loss = 235.408524, Hit@1 = 0.63, PERR = 0.63, GAP = 0.72 [INFO: metrics_util.py: 124]: [TEST] Epoch2 Finish avg_hit_at_one: 0.6002604166666666, avg_perr: 0.6002604166666666, avg_loss :271.76447041829425, aps: [0.31461424520869374, 0.8936133494848872, 0.11043648749825534, 0.125, 0.03722438391699092, 0.8856478710931908, 0.6146493900189034, 0.22876525268503656, 0.10540428561979417, 0.8973407150987074, 0.07046950416809067, 1.0, 0.5973890927620836, 0.19951584164627645, 0.6677338493633069, 0.5268199460078398, 0.24872847879439597, 0.8782532773887733, 0.1096882518043304, 0.8638357292555305, 0.5653448812511401, 0.32151854978205635, 0.7386509578503802, 0, 0.2089245778417543, 0.36278048502308347, 0.9742929999991989, 0.25464991632445033, 0.13271150047535973, 0.13510689282158658, 0.0, 0.04718878169155384, 0.6798868023231818], gap:0.6512923596529407 [INFO: regularizer.py: 101]: If regularizer of a Parameter has been set by 'fluid.ParamAttr' or 'fluid.WeightNormParamAttr' already. The Regularization[L2Decay, regularization_coeff=0.000800] in Optimizer will not take effect, and it will only be applied to other Parameters! [INFO: attention_lstm.py: 164]: Load pretrain weights from /home/aistudio/work/paddle-video-classify-tag/weights/attention_lstm, exclude fc layer. [INFO: train_utils.py: 45]: ------- learning rate [0.000125], learning rate counter [-] ----- [INFO: metrics_util.py: 80]: [TRAIN 2022-04-28 22:13:03] Epoch 0, iter 0, time 2.3079347610473633, , loss = 24682.550781, Hit@1 = 0.00, PERR = 0.00, GAP = 0.00 [INFO: train_utils.py: 122]: [TRAIN] Epoch 0 training finished, average time: 1.7917025527175592 share_vars_from is set, scope is ignored. [INFO: metrics_util.py: 80]: [TEST] test_iter 0 , loss = 600.804932, Hit@1 = 0.30, PERR = 0.30, GAP = 0.24 [INFO: metrics_util.py: 124]: [TEST] Epoch0 Finish avg_hit_at_one: 0.21549479166666666, avg_perr: 0.21549479166666666, avg_loss :640.4301401774088, aps: [0, 0, 0, 0.40242828226797467, 0, 0, 0, 0, 0.33863014399927605, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.017490013599745342, 0, 0, 0, 0, 0, 0, 0, 0.03521126486951533, 0, 0, 0, 0, 0, 0, 0.03405457205661094, 0, 0, 0, 0, 0.0, 0.6962887915273482, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.04010896770467283, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.05729046027105949, 0, 0, 0, 0, 0.006493506493506493, 0, 0, 0, 0, 0, 0.6447801522817966, 0, 0, 0.06971153846153846, 0, 0, 0, 0, 0, 0.06551067886636681, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.20577600830312404, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.05182387738056744, 0.19935288798575898, 0, 0, 0, 0, 0, 0, 0.006060606060606061, 0, 0.01652246148167635, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.3408273189259355, 0, 0, 0.22446935557021744, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.10661600278272008, 0, 0, 0, 0.03607677045177045, 0, 0, 0, 0, 0.12624295024072427, 0, 0, 0, 0.011908733985379907, 0, 0, 0, 0, 0, 0.01123077136610415, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.14973782613585243, 0, 0, 0.6419203037914082, 0.3315040696329643, 0, 0, 0, 0.03735729886111951, 0, 0, 0, 0, 0.0006313131313131313, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.2578085880759878, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], gap:0.1536302281405276 [INFO: train_utils.py: 45]: ------- learning rate [0.000125], learning rate counter [-] ----- [INFO: metrics_util.py: 80]: [TRAIN 2022-04-28 22:15:21] Epoch 1, iter 0, time 2.288398265838623, , loss = 643.896606, Hit@1 = 0.26, PERR = 0.26, GAP = 0.17 [INFO: train_utils.py: 122]: [TRAIN] Epoch 1 training finished, average time: 1.7937103728858792 [INFO: metrics_util.py: 80]: [TEST] test_iter 0 , loss = 506.925232, Hit@1 = 0.42, PERR = 0.42, GAP = 0.35 [INFO: metrics_util.py: 124]: [TEST] Epoch1 Finish avg_hit_at_one: 0.322265625, avg_perr: 0.322265625, avg_loss :543.9810791015625, aps: [0, 0.6284037777940216, 0, 0.4476176894075993, 0, 0.017502088554720133, 0, 0.0, 0.49948578553903844, 0.6726278000591354, 0, 0, 0.014814814814814815, 0, 0, 0.0, 0, 0, 0.12025469917857523, 0, 0, 0, 0, 0, 0, 0, 0.06116341245294592, 0, 0.0, 0, 0, 0, 0, 0.08986510263555623, 0, 0, 0, 0.08181818181818182, 0.1834875432956615, 0.7799351937440513, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.8018812602440357, 0, 0, 0, 0, 0.0, 0, 0, 0, 0, 0, 0, 0.03716972682489924, 0, 0, 0, 0.35282327653333223, 0.29747338431548953, 0, 0, 0, 0.03269644204099986, 0, 0, 0, 0.10658710658710657, 0, 0.7397457184599068, 0.021230242805867577, 0.03970086959790015, 0.21014492753623187, 0, 0, 0, 0, 0, 0.2012167613391781, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 0, 0, 0, 0, 0.5718171721543543, 0.20836023284422855, 0, 0, 0.0205176753138432, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 0, 0.32366077494535755, 0.5132305645871157, 0, 0, 0, 0, 0, 0, 0.059059953550699054, 0, 0.02679003573313643, 0, 0, 0, 0, 0.010724231560075813, 0, 0, 0.0, 0, 0.3694164859158865, 0.04969704265478912, 0.06605492031500655, 0.26678914407765286, 0.5, 0.1117114831313118, 0.48005756650486286, 0, 0, 0, 0, 0, 0, 0, 0.020842379504993486, 0, 0, 0, 0.16806455477716875, 0, 0, 0, 0.35504129480409174, 0, 0, 0, 0, 0.18455439535372678, 0, 0.10785929617324966, 0, 0.07077634245187436, 0, 0, 0, 0.15618791293239848, 0, 0.013192995678206728, 0, 0, 0, 0, 0, 0, 0.38888888888888895, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.9289044289044288, 0.38066141949737636, 0, 0, 0.7315730501252811, 0.30272837068177305, 0.03046189453320919, 0, 0, 0.19456917878558563, 0, 0, 0, 0, 0.015130219289343896, 0, 0, 0, 0, 0, 0, 0, 0, 0.03746427683108207, 0, 0, 0, 0.1412861620299421, 0, 0, 0, 0, 0, 0, 0, 0.3944251458547141, 0, 0, 0.2678798532334657, 0.5851648351648351, 0, 0, 0, 0.018914641343419712, 0, 0, 0.12094981620223223, 0.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], gap:0.2629719332516419 [INFO: train_utils.py: 45]: ------- learning rate [0.000125], learning rate counter [-] ----- [INFO: metrics_util.py: 80]: [TRAIN 2022-04-28 22:17:41] Epoch 2, iter 0, time 2.2935118675231934, , loss = 554.811279, Hit@1 = 0.31, PERR = 0.31, GAP = 0.25 [INFO: train_utils.py: 122]: [TRAIN] Epoch 2 training finished, average time: 1.8651155549652723 [INFO: metrics_util.py: 80]: [TEST] test_iter 0 , loss = 457.075806, Hit@1 = 0.42, PERR = 0.42, GAP = 0.43 [INFO: metrics_util.py: 124]: [TEST] Epoch2 Finish avg_hit_at_one: 0.3645833333333333, avg_perr: 0.3645833333333333, avg_loss :490.911376953125, aps: [0, 0.7264839860993707, 0, 0.47462585341997715, 0, 0.014711590709519537, 0, 0.04822073388816287, 0.5609560108708409, 0.8027352365113277, 0, 0, 0.32129792215393616, 0, 0, 0.02431476569407604, 0, 0.0, 0.2461927562794056, 0, 0, 0, 0, 0, 0.1660546382768605, 0.03484848484848485, 0.06536851469684964, 0, 0.0, 0, 0, 0, 0, 0.08563124068799817, 0, 0, 0, 0.34035087719298246, 0.33772339731642054, 0.8509592092451618, 0, 0.03125, 0, 0.0, 0.020867208672086725, 0.039261363636363636, 0, 0, 0, 0, 0.8568407053083421, 0, 0, 0, 0, 0.03447038958508859, 0, 0, 0, 0.004165394142997726, 0, 0.08268743667679837, 0.048576534576534575, 0, 0, 0, 0.41189811345296407, 0.6160275221199591, 0, 0, 0, 0.05479598297850816, 0, 0.011745038316747055, 0, 0.14839572192513367, 0.017857142857142856, 0.8142012317112416, 0.03919328856298826, 0.10154059203661106, 0.5147058823529411, 0, 0.0027100271002710027, 0.0, 0, 0, 0.25578997486755245, 0, 0.0, 0.03891402714932127, 0, 0.16742424242424242, 0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0.6971533526064891, 0.25392683350632306, 0.0, 0, 0.12435782848299608, 0, 0, 0.0, 0, 0, 0, 0, 0, 0.03965336134453781, 0, 0.3146511903607467, 0.5880597551385015, 0, 0, 0, 0, 0, 0, 0.08627606708336637, 0, 0.031046668257497162, 0, 0, 0, 0, 0.012812752669984209, 0, 0, 0.027443216190947587, 0, 0.3842286813576051, 0.3885135135135135, 0.08781870642994039, 0.6136011372304279, 0.10267857142857142, 0.17066271855227885, 0.5040137737506158, 0, 0, 0, 0, 0.0, 0, 0.018518518518518517, 0.12702702702702703, 0, 0.09607023411371238, 0, 0.2145263036917232, 0, 0, 0, 0.48504451428566525, 0, 0, 0, 0, 0.24584729615785442, 0, 0.18667368667368664, 0, 0.13666142557651992, 0, 0, 0, 0.3403490239016555, 0.7, 0.025737892154730938, 0.030303030303030304, 0, 0, 0, 0, 0, 0.7166666666666667, 0, 0, 0, 0, 0, 0.027777777777777776, 0.15572533484248255, 0, 0, 0.9294127880666342, 0.3886217948717949, 0, 0, 0.8316227550780863, 0.3991161073245167, 0.13076985608832742, 0, 0, 0.2779068113777733, 0, 0, 0.0, 0, 0.023388721455443927, 0, 0, 0, 0, 0, 0, 0, 0, 0.08238636363636363, 0, 0.0, 0, 0.2810910550416964, 0, 0.06020066889632107, 0, 0, 0, 0, 0, 0.47000346625667766, 0, 0, 0.4060489971691195, 0.6409310594991161, 0, 0, 0, 0.027279501544207423, 0, 0, 0.2954107030662055, 0.1875, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], gap:0.35114517815351143

2.3.3 生成分類標(biāo)簽結(jié)果

運(yùn)行以下代碼塊生成標(biāo)簽預(yù)測結(jié)果。

生成的標(biāo)簽結(jié)果存儲(chǔ)在./predict_results/level{1, 2}_top1.json

In [1]

import os import sys import time import logging import argparse import ast import numpy as np import paddle try:import cPickle as pickle except:import picklesys.path.append('/home/aistudio/external-libraries') sys.path.append('/home/aistudio/work/paddle-video-classify-tag')from utils.config_utils import * import models from reader import get_reader from metrics import get_metrics from utils.utility import check_cuda from utils.utility import check_versionlogging.root.handlers = [] FORMAT = '[%(levelname)s: %(filename)s: %(lineno)4d]: %(message)s' logging.basicConfig(level=logging.DEBUG, format=FORMAT) logger = logging.getLogger(__name__)class Args():model_name = 'AttentionLSTM'config = '/home/aistudio/work/paddle-video-classify-tag/configs/attention_lstm-single-level1.yaml'use_gpu = Truebatch_size = Noneweights = '/home/aistudio/paddle-video-classify-tag/data/checkpoints/level1/AttentionLSTM_epoch2.pdparams'filelist = Nonelog_interval = 200infer_topk = 10save_dir = './predict_results'save_file = "top1.json"label_file = '/home/aistudio/paddle-video-classify-tag/data/level1_label.txt'video_path = Nonedef infer(args):# parse configconfig = parse_config(args.config)infer_config = merge_configs(config, 'infer', vars(args))print_configs(infer_config, "Infer")infer_model = models.get_model(args.model_name, infer_config, mode='infer')infer_model.build_input(use_dataloader=False)infer_model.build_model()infer_feeds = infer_model.feeds()infer_outputs = infer_model.outputs()place = paddle.CUDAPlace(0) if args.use_gpu else paddle.CPUPlace()exe = paddle.static.Executor(place)exe.run(paddle.static.default_startup_program())filelist = args.filelist or infer_config.INFER.filelistfilepath = args.video_path or infer_config.INFER.get('filepath', '')if filepath != '':assert os.path.exists(filepath), "{} not exist.".format(filepath)else:assert os.path.exists(filelist), "{} not exist.".format(filelist)# get infer readerinfer_reader = get_reader(args.model_name.upper(), 'infer', infer_config)if args.weights:assert os.path.exists(args.weights), "Given weight dir {} not exist.".format(args.weights)# if no weight files specified, download weights from paddleweights = args.weights or infer_model.get_weights()infer_model.load_test_weights(exe, weights,paddle.static.default_main_program())infer_feeder = paddle.fluid.DataFeeder(place=place, feed_list=infer_feeds)fetch_list = infer_model.fetches()infer_metrics = get_metrics(args.model_name.upper(), 'infer', infer_config)infer_metrics.reset()periods = []cur_time = time.time()for infer_iter, data in enumerate(infer_reader()):data_feed_in = [items[:-1] for items in data]video_id = [items[-1] for items in data]infer_outs = exe.run(fetch_list=fetch_list,feed=infer_feeder.feed(data_feed_in))infer_result_list = [item for item in infer_outs] + [video_id]prev_time = cur_timecur_time = time.time()period = cur_time - prev_timeperiods.append(period)infer_metrics.accumulate(infer_result_list)if args.log_interval > 0 and infer_iter % args.log_interval == 0:logger.info('Processed {} samples'.format((infer_iter + 1) * len(video_id)))logger.info('[INFER] infer finished. average time: {}'.format(np.mean(periods)))if not os.path.isdir(args.save_dir):os.makedirs(args.save_dir)infer_metrics.finalize_and_log_out(savedir=args.save_dir,savefile=args.save_file,label_file=args.label_file)args = Args()# 一級(jí)標(biāo)簽 args.config = '/home/aistudio/work/paddle-video-classify-tag/configs/attention_lstm-single-level1.yaml' args.weights = '/home/aistudio/paddle-video-classify-tag/data/checkpoints/level1/AttentionLSTM_epoch2.pdparams' args.label_file = '/home/aistudio/paddle-video-classify-tag/data/level1_label.txt' args.save_file = 'level1_top1.json' infer(args)# 二級(jí)標(biāo)簽 args.config = '/home/aistudio/work/paddle-video-classify-tag/configs/attention_lstm-single-level2.yaml' args.weights = '/home/aistudio/paddle-video-classify-tag/data/checkpoints/level2/AttentionLSTM_epoch2.pdparams' args.label_file = '/home/aistudio/paddle-video-classify-tag/data/level2_label.txt' args.save_file = 'level2_top1.json' infer(args) /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/utils.py:26: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecationsdef convert_to_list(value, n, name, dtype=np.int): [INFO: config_utils.py: 70]: ---------------- Infer Arguments ---------------- [INFO: config_utils.py: 72]: MODEL: [INFO: config_utils.py: 74]: name:AttentionLSTM [INFO: config_utils.py: 74]: dataset:YouTube-8M [INFO: config_utils.py: 74]: bone_nework:None [INFO: config_utils.py: 74]: drop_rate:0.5 [INFO: config_utils.py: 74]: feature_names:['rgb'] [INFO: config_utils.py: 74]: feature_dims:[2048] [INFO: config_utils.py: 74]: embedding_size:1024 [INFO: config_utils.py: 74]: lstm_size:512 [INFO: config_utils.py: 74]: num_classes:278 [INFO: config_utils.py: 74]: topk:20 [INFO: config_utils.py: 72]: TRAIN: [INFO: config_utils.py: 74]: epoch:3 [INFO: config_utils.py: 74]: learning_rate:0.000125 [INFO: config_utils.py: 74]: decay_epochs:[5] [INFO: config_utils.py: 74]: decay_gamma:0.1 [INFO: config_utils.py: 74]: weight_decay:0.0008 [INFO: config_utils.py: 74]: num_samples:35952 [INFO: config_utils.py: 74]: pretrain_base:None [INFO: config_utils.py: 74]: batch_size:128 [INFO: config_utils.py: 74]: use_gpu:True [INFO: config_utils.py: 74]: num_gpus:1 [INFO: config_utils.py: 74]: filelist:/home/aistudio/paddle-video-classify-tag/data/level2_train.list [INFO: config_utils.py: 72]: VALID: [INFO: config_utils.py: 74]: batch_size:128 [INFO: config_utils.py: 74]: filelist:/home/aistudio/paddle-video-classify-tag/data/level2_val.list [INFO: config_utils.py: 72]: TEST: [INFO: config_utils.py: 74]: batch_size:128 [INFO: config_utils.py: 74]: filelist:/home/aistudio/paddle-video-classify-tag/data/level2_val.list [INFO: config_utils.py: 72]: INFER: [INFO: config_utils.py: 74]: batch_size:1 [INFO: config_utils.py: 74]: filelist:/home/aistudio/paddle-video-classify-tag/data/level2_test.list [INFO: config_utils.py: 75]: ------------------------------------------------- W0428 23:33:54.649835 8407 device_context.cc:362] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.0, Runtime API Version: 10.1 W0428 23:33:54.655040 8407 device_context.cc:372] device: 0, cuDNN Version: 7.6. [INFO: 3560385804.py: 106]: Processed 1 samples [INFO: 3560385804.py: 106]: Processed 201 samples [INFO: 3560385804.py: 106]: Processed 401 samples [INFO: 3560385804.py: 106]: Processed 601 samples [INFO: 3560385804.py: 106]: Processed 801 samples [INFO: 3560385804.py: 106]: Processed 1001 samples [INFO: 3560385804.py: 106]: Processed 1201 samples [INFO: 3560385804.py: 106]: Processed 1401 samples [INFO: 3560385804.py: 106]: Processed 1601 samples [INFO: 3560385804.py: 106]: Processed 1801 samples [INFO: 3560385804.py: 106]: Processed 2001 samples [INFO: 3560385804.py: 106]: Processed 2201 samples [INFO: 3560385804.py: 106]: Processed 2401 samples [INFO: 3560385804.py: 106]: Processed 2601 samples [INFO: 3560385804.py: 106]: Processed 2801 samples [INFO: 3560385804.py: 106]: Processed 3001 samples [INFO: 3560385804.py: 106]: Processed 3201 samples [INFO: 3560385804.py: 106]: Processed 3401 samples [INFO: 3560385804.py: 106]: Processed 3601 samples [INFO: 3560385804.py: 106]: Processed 3801 samples [INFO: 3560385804.py: 106]: Processed 4001 samples [INFO: 3560385804.py: 106]: Processed 4201 samples [INFO: 3560385804.py: 106]: Processed 4401 samples [INFO: 3560385804.py: 106]: Processed 4601 samples [INFO: 3560385804.py: 106]: Processed 4801 samples [INFO: 3560385804.py: 106]: Processed 5001 samples [INFO: 3560385804.py: 106]: Processed 5201 samples [INFO: 3560385804.py: 106]: Processed 5401 samples [INFO: 3560385804.py: 106]: Processed 5601 samples [INFO: 3560385804.py: 106]: Processed 5801 samples [INFO: 3560385804.py: 106]: Processed 6001 samples [INFO: 3560385804.py: 106]: Processed 6201 samples [INFO: 3560385804.py: 106]: Processed 6401 samples [INFO: 3560385804.py: 106]: Processed 6601 samples [INFO: 3560385804.py: 106]: Processed 6801 samples [INFO: 3560385804.py: 106]: Processed 7001 samples [INFO: 3560385804.py: 106]: Processed 7201 samples [INFO: 3560385804.py: 106]: Processed 7401 samples [INFO: 3560385804.py: 106]: Processed 7601 samples [INFO: 3560385804.py: 106]: Processed 7801 samples [INFO: 3560385804.py: 106]: Processed 8001 samples [INFO: 3560385804.py: 106]: Processed 8201 samples [INFO: 3560385804.py: 106]: Processed 8401 samples [INFO: 3560385804.py: 106]: Processed 8601 samples [INFO: 3560385804.py: 106]: Processed 8801 samples [INFO: 3560385804.py: 106]: Processed 9001 samples [INFO: 3560385804.py: 106]: Processed 9201 samples [INFO: 3560385804.py: 106]: Processed 9401 samples [INFO: 3560385804.py: 106]: Processed 9601 samples [INFO: 3560385804.py: 106]: Processed 9801 samples [INFO: 3560385804.py: 109]: [INFER] infer finished. average time: 0.03070716172327805 [INFO: metrics_util.py: 119]: Saved ./predict_results/level2_top1.json

2.4 視頻語義標(biāo)簽基線

該基線提供視頻語義標(biāo)簽的理解能力，基線從視頻的文本信息中抽取表示視頻內(nèi)容主旨的語義標(biāo)簽知識(shí)（選手可進(jìn)行升級(jí)，如利用給定的知識(shí)庫進(jìn)行推理、融合多模信息提升標(biāo)簽理解效果等生成標(biāo)簽）。

2.4.1 數(shù)據(jù)處理

首先將數(shù)據(jù)整理成命名實(shí)體識(shí)別模型所需格式，并劃分訓(xùn)練集、驗(yàn)證集等?？梢詤⒖糚addleNLP中文命名實(shí)體項(xiàng)目。

注：我們?cè)跀?shù)據(jù)處理階段去除了未在title中出現(xiàn)的語義標(biāo)簽。

In [8]

import os import pandas as pd import json import codecs import argparse import randomsys.path.append('/home/aistudio/external-libraries') TAG_NAMES = ["B-ENT", "I-ENT", "O"]class Args():trainval_path = '/home/aistudio/dataset_sample/train.sample.json'test_path = '/home/aistudio/dataset_sample/test_a.json'def gather_text_and_tags(sample, test_only=False):def fill_tags(surf):'''For entities that appear in text, replace their tags with 'B-ENT/I-ENT'.'''s_idx = text.find(surf)if s_idx != -1:tags[s_idx] = TAG_NAMES[0]for i in range(s_idx + 1, s_idx + len(surf)):tags[i] = TAG_NAMES[1]return 1return 0text = sample["title"].replace(" ", "").replace("\t", "")# init tag sequence with all 'O's.tags = [TAG_NAMES[2] for i in range(len(text))]entities = []if not test_only:entities = [each["@value"] for each in sample["tag"]]# annotate 'B-ENT' and 'I-ENT' tags.n_bingo_entities = sum([fill_tags(surf) for surf in entities if len(surf) > 0])# statisticsstats = {"txt_length": len(text),"n_entities": len(entities),"n_bingo_entities": n_bingo_entities,}return text, tags, statsdef stat_numberic_list(li, name="default"):assert isinstance(li, list)stat = {}stat["size"] = len(li)if all(isinstance(x, int) for x in li):stat["max"] = max(li)stat["min"] = min(li)stat["sum"] = sum(li)stat["avr"] = stat["sum"] / float(len(li))print("list-%s:\n\t%s" % (name, str(stat)))def analyze_annots(stats_list):for key in ["txt_length", "n_entities", "n_bingo_entities"]:numbers = [stats[key] for stats in stats_list]stat_numberic_list(numbers, name=key)def prepare_split(data, split_name, test_only=False):sample_lines = []nid_lines = []stats_list = []for idx in range(len(data)):text, tags, stats = gather_text_and_tags(data[idx], test_only=test_only)if len(text) == 0:continue# proper data format.text = '\002'.join([ch for ch in text])tags = '\002'.join(tags)sample_lines.append('\t'.join([text, tags]) + "\n")nid_lines.append(data[idx]["@id"] + "\n")stats_list.append(stats)if split_name == "trainval":# print statistics.analyze_annots(stats_list)save_split_file = "/home/aistudio/paddle-video-semantic-tag/data/{}.tsv".format(split_name)with codecs.open(save_split_file, "w", encoding="utf-8") as ouf:ouf.writelines(sample_lines)print("Saved {}, size={}".format(save_split_file, len(sample_lines)))with codecs.open("/home/aistudio/paddle-video-semantic-tag/data/nids.txt", "w", encoding="utf-8") as ouf:ouf.writelines(nid_lines)def create_splits_indice(n_samples, SPLITS):assert sum([v for k, v in SPLITS]) == 1.0indices = list(range(n_samples))random.shuffle(indices)split2indice = {}r_offset = 0for idx, (split, ratio) in enumerate(SPLITS):l_offset = r_offsetif idx == len(SPLITS) - 1:r_offset = n_sampleselse:r_offset = int(n_samples * ratio) + l_offsetsplit2indice[split] = indices[l_offset:r_offset]return split2indiceargs = Args() random.seed(6666)# load data for train & validation (have labels). with codecs.open(args.trainval_path, "r", encoding="utf-8") as inf:print("Loading {}...".format(args.trainval_path))lines = inf.readlines()trainval_data = [json.loads(line) for line in lines]# load data for test (no labels). with codecs.open(args.test_path, "r", encoding="utf-8") as inf:print("Loading {}...".format(args.test_path))lines = inf.readlines()test_data = [json.loads(line) for line in lines]# split the trainval data into train-set(80%) and validation-set(20%). split2indice = create_splits_indice(len(trainval_data), [("train", 3.0 / 4.0),("val", 1.0 / 4.0),]) train_data = [trainval_data[idx] for idx in split2indice["train"]] val_data = [trainval_data[idx] for idx in split2indice["val"]]label_map_file = "/home/aistudio/paddle-video-semantic-tag/data/label_map.json" with open(label_map_file, "w") as ouf:json.dump({tag: idx for idx, tag in enumerate(TAG_NAMES)}, ouf) print("Saved " + label_map_file)prepare_split(trainval_data, "trainval") prepare_split(train_data, "train") prepare_split(val_data, "val") prepare_split(test_data, "test", test_only=True) Loading /home/aistudio/dataset_sample/train.sample.json... Loading /home/aistudio/dataset_sample/test_a.json... Saved /home/aistudio/paddle-video-semantic-tag/data/label_map.json list-txt_length:{'size': 8079, 'max': 67, 'min': 1, 'sum': 118072, 'avr': 14.614680034657754} list-n_entities:{'size': 8079, 'max': 38, 'min': 0, 'sum': 21425, 'avr': 2.6519371209308082} list-n_bingo_entities:{'size': 8079, 'max': 8, 'min': 0, 'sum': 8292, 'avr': 1.0263646490902338} Saved /home/aistudio/paddle-video-semantic-tag/data/trainval.tsv, size=8079 Saved /home/aistudio/paddle-video-semantic-tag/data/train.tsv, size=6061 Saved /home/aistudio/paddle-video-semantic-tag/data/val.tsv, size=2018 Saved /home/aistudio/paddle-video-semantic-tag/data/test.tsv, size=9938

2.3.3 訓(xùn)練與驗(yàn)證

本模型使用了PaddleNLP模型庫中的bert-wwm-ext-chinese模型，更多模型可參考PaddleNLP Transformer API。

In [?]

import argparse import os import sys import random import time import math from functools import partial import jsonimport numpy as np import paddle from paddle.io import DataLoaderimport paddlenlp as ppnlp from paddlenlp.transformers import LinearDecayWithWarmup from paddlenlp.metrics import ChunkEvaluator from paddlenlp.datasets import load_dataset from paddlenlp.transformers import BertForTokenClassification, BertTokenizer from paddlenlp.data import Stack, Tuple, Pad, Dictsys.path.append('/home/aistudio/external-libraries')class Args():model_name_or_path = Noneoutput_dir = Nonemax_seq_length = 128batch_size = 8learning_rate = 5e-5weight_decay = 0.0adam_epsilon = 1e-8max_grad_norm = 1.0num_train_epochs = 3max_steps = -1warmup_steps = 0logging_steps = 1save_steps = 100seed = 42device = 'gpu'def evaluate(model, loss_fct, metric, data_loader, label_num):model.eval()metric.reset()avg_loss, precision, recall, f1_score = 0, 0, 0, 0for batch in data_loader:input_ids, token_type_ids, length, labels = batchlogits = model(input_ids, token_type_ids)loss = loss_fct(logits, labels)avg_loss = paddle.mean(loss)preds = logits.argmax(axis=2)num_infer_chunks, num_label_chunks, num_correct_chunks = metric.compute(None, length, preds, labels)metric.update(num_infer_chunks.numpy(),num_label_chunks.numpy(), num_correct_chunks.numpy())precision, recall, f1_score = metric.accumulate()print("eval loss: %f, precision: %f, recall: %f, f1: %f" %(avg_loss, precision, recall, f1_score))model.train()def tokenize_and_align_labels(example,tokenizer,no_entity_id,max_seq_len=512):labels = example['labels']example = example['tokens']tokenized_input = tokenizer(example,return_length=True,is_split_into_words=True,max_seq_len=max_seq_len)# -2 for [CLS] and [SEP]if len(tokenized_input['input_ids']) - 2 < len(labels):labels = labels[:len(tokenized_input['input_ids']) - 2]tokenized_input['labels'] = [no_entity_id] + labels + [no_entity_id]tokenized_input['labels'] += [no_entity_id] * (len(tokenized_input['input_ids']) - len(tokenized_input['labels']))return tokenized_inputdef _read(data_file, label_map_file):with open(label_map_file, "r") as inf:tag2label = json.load(inf)with open(data_file, 'r', encoding='utf-8') as inf:for line in inf:line_stripped = line.strip().split('\t')assert len(line_stripped) == 2tokens = line_stripped[0].split("\002")tags = line_stripped[1].split("\002")labels = [tag2label[tag] for tag in tags]yield {"tokens": tokens, "labels": labels}def do_train(args):paddle.set_device(args.device)if paddle.distributed.get_world_size() > 1:paddle.distributed.init_parallel_env()# Create dataset, tokenizer and dataloader.train_ds = load_dataset(_read,data_file="/home/aistudio/paddle-video-semantic-tag/data/train.tsv",label_map_file="/home/aistudio/paddle-video-semantic-tag/data/label_map.json",lazy=False)test_ds = load_dataset(_read,data_file="/home/aistudio/paddle-video-semantic-tag/data/val.tsv",label_map_file="/home/aistudio/paddle-video-semantic-tag/data/label_map.json",lazy=False)train_ds.label_list = test_ds.label_list = ["B-ENT", "I-ENT", "O"]tokenizer = BertTokenizer.from_pretrained(args.model_name_or_path)label_list = train_ds.label_listlabel_num = len(label_list)no_entity_id = label_num - 1trans_func = partial(tokenize_and_align_labels,tokenizer=tokenizer,no_entity_id=no_entity_id,max_seq_len=args.max_seq_length)train_ds = train_ds.map(trans_func)ignore_label = -100batchify_fn = lambda samples, fn=Dict({'input_ids': Pad(axis=0, pad_val=tokenizer.pad_token_id), # input'token_type_ids': Pad(axis=0, pad_val=tokenizer.pad_token_type_id), # segment'seq_len': Stack(), # seq_len'labels': Pad(axis=0, pad_val=ignore_label) # label}): fn(samples)train_batch_sampler = paddle.io.DistributedBatchSampler(train_ds, batch_size=args.batch_size, shuffle=True, drop_last=True)train_data_loader = DataLoader(dataset=train_ds,collate_fn=batchify_fn,num_workers=0,batch_sampler=train_batch_sampler,return_list=True)test_ds = test_ds.map(trans_func)test_data_loader = DataLoader(dataset=test_ds,collate_fn=batchify_fn,num_workers=0,batch_size=args.batch_size,return_list=True)# Define the model netword and its lossmodel = BertForTokenClassification.from_pretrained(args.model_name_or_path, num_classes=label_num)if paddle.distributed.get_world_size() > 1:model = paddle.DataParallel(model)num_training_steps = args.max_steps if args.max_steps > 0 else len(train_data_loader) * args.num_train_epochslr_scheduler = LinearDecayWithWarmup(args.learning_rate,num_training_steps, args.warmup_steps)# Generate parameter names needed to perform weight decay.# All bias and LayerNorm parameters are excluded.decay_params = [p.name for n, p in model.named_parameters()if not any(nd in n for nd in ["bias", "norm"])]optimizer = paddle.optimizer.AdamW(learning_rate=lr_scheduler,epsilon=args.adam_epsilon,parameters=model.parameters(),weight_decay=args.weight_decay,apply_decay_param_fun=lambda x: x in decay_params)loss_fct = paddle.nn.loss.CrossEntropyLoss(ignore_index=ignore_label)metric = ChunkEvaluator(label_list=label_list)global_step = 0last_step = args.num_train_epochs * len(train_data_loader)tic_train = time.time()for epoch in range(args.num_train_epochs):for step, batch in enumerate(train_data_loader):global_step += 1input_ids, token_type_ids, _, labels = batchlogits = model(input_ids, token_type_ids)loss = loss_fct(logits, labels)avg_loss = paddle.mean(loss)if global_step % args.logging_steps == 0:print("global step %d, epoch: %d, batch: %d, loss: %f, speed: %.2f step/s"% (global_step, epoch, step, avg_loss,args.logging_steps / (time.time() - tic_train)))tic_train = time.time()avg_loss.backward()optimizer.step()lr_scheduler.step()optimizer.clear_grad()if global_step % args.save_steps == 0 or global_step == last_step:if paddle.distributed.get_rank() == 0:evaluate(model, loss_fct, metric, test_data_loader,label_num)paddle.save(model.state_dict(),os.path.join(args.output_dir,"model_%d.pdparams" % global_step))args = Args() args.model_name_or_path = 'bert-wwm-ext-chinese' args.max_seq_length = 128 args.batch_size = 32 args.learning_rate = 2e-5 args.num_train_epochs = 3 args.logging_steps = 1 args.save_steps = 500 args.output_dir = '/home/aistudio/paddle-video-semantic-tag/data/checkpoints/semantic_tag' args.device = 'gpu' do_train(args)

In [4]

!ls /home/aistudio/paddle-video-semantic-tag/data/checkpoints/se* model_500.pdparams model_567.pdparams

2.3.4 生成語義標(biāo)簽結(jié)果

生成的識(shí)別結(jié)果存儲(chǔ)在./predict_results/ents_results.json。

In [5]

import argparse import os import sys import ast import random import time import math from functools import partial import json import codecs from tqdm import tqdmimport numpy as np import paddle from paddle.io import DataLoaderimport paddlenlp as ppnlp from paddlenlp.datasets import load_dataset from paddlenlp.data import Stack, Tuple, Pad, Dict from paddlenlp.transformers import BertForTokenClassification, BertTokenizersys.path.append('/home/aistudio/external-libraries')class Args():model_name_or_path = Noneoutput_dir = Nonemax_seq_length = 128batch_size = 8learning_rate = 5e-5weight_decay = 0.0adam_epsilon = 1e-8max_grad_norm = 1.0num_train_epochs = 3max_steps = -1warmup_steps = 0logging_steps = 1save_steps = 100seed = 42device = 'gpu'def tokenize_and_align_labels(example,tokenizer,no_entity_id,max_seq_len=512):labels = example['labels']example = example['tokens']tokenized_input = tokenizer(example,return_length=True,is_split_into_words=True,max_seq_len=max_seq_len)# -2 for [CLS] and [SEP]if len(tokenized_input['input_ids']) - 2 < len(labels):labels = labels[:len(tokenized_input['input_ids']) - 2]tokenized_input['labels'] = [no_entity_id] + labels + [no_entity_id]tokenized_input['labels'] += [no_entity_id] * (len(tokenized_input['input_ids']) - len(tokenized_input['labels']))return tokenized_inputdef parse_decodes(input_words, id2label, decodes, lens):decodes = [x for batch in decodes for x in batch]lens = [x for batch in lens for x in batch]outputs = []entities_list = []for idx, end in enumerate(lens):sent = "".join(input_words[idx]['tokens'])tags = [id2label[x] for x in decodes[idx][1:end]]sent_out = []tags_out = []words = ""for s, t in zip(sent, tags):if t.startswith('B-') or t == 'O':if len(words):sent_out.append(words)if t.startswith('B-'):tags_out.append(t.split('-')[1])else:tags_out.append(t)words = selse:words += sif len(sent_out) < len(tags_out):sent_out.append(words)outputs.append(''.join([str((s, t)) for s, t in zip(sent_out, tags_out)]))entities_list.append([s for s, t in zip(sent_out, tags_out) if t == "ENT"])return outputs, entities_listdef _read(data_file, label_map_file):with open(label_map_file, "r") as inf:tag2label = json.load(inf)with open(data_file, 'r', encoding='utf-8') as inf:for line in inf:line_stripped = line.strip().split('\t')assert len(line_stripped) == 2tokens = line_stripped[0].split("\002")tags = line_stripped[1].split("\002")labels = [tag2label[tag] for tag in tags]yield {"tokens": tokens, "labels": labels}def do_predict(args):paddle.set_device(args.device)# Create dataset, tokenizer and dataloader.train_ds = load_dataset(_read,data_file="/home/aistudio/paddle-video-semantic-tag/data/train.tsv",label_map_file="/home/aistudio/paddle-video-semantic-tag/data/label_map.json",lazy=False)predict_ds = load_dataset(_read,data_file="/home/aistudio/paddle-video-semantic-tag/data/test.tsv",label_map_file="/home/aistudio/paddle-video-semantic-tag/data/label_map.json",lazy=False)train_ds.label_list = predict_ds.label_list = ["B-ENT", "I-ENT", "O"]tokenizer = BertTokenizer.from_pretrained(args.model_name_or_path)label_list = train_ds.label_listlabel_num = len(label_list)no_entity_id = label_num - 1trans_func = partial(tokenize_and_align_labels,tokenizer=tokenizer,no_entity_id=no_entity_id,max_seq_len=args.max_seq_length)ignore_label = -100batchify_fn = lambda samples, fn=Dict({'input_ids': Pad(axis=0, pad_val=tokenizer.pad_token_id), # input'token_type_ids': Pad(axis=0, pad_val=tokenizer.pad_token_type_id), # segment'seq_len': Stack(),'labels': Pad(axis=0, pad_val=ignore_label) # label}): fn(samples)raw_data = predict_ds.dataid2label = dict(enumerate(predict_ds.label_list))predict_ds = predict_ds.map(trans_func)predict_data_loader = DataLoader(dataset=predict_ds,collate_fn=batchify_fn,num_workers=0,batch_size=args.batch_size,return_list=True)# Define the model networdmodel = BertForTokenClassification.from_pretrained(args.model_name_or_path, num_classes=label_num)if args.init_checkpoint_path:model_dict = paddle.load(args.init_checkpoint_path)model.set_dict(model_dict)model.eval()pred_list = []len_list = []for step, batch in tqdm(enumerate(predict_data_loader)):input_ids, token_type_ids, length, labels = batchlogits = model(input_ids, token_type_ids)pred = paddle.argmax(logits, axis=-1)pred_list.append(pred.numpy())len_list.append(length.numpy())preds, entities_list = parse_decodes(raw_data, id2label, pred_list,len_list)save_dir = "predict_results"if not os.path.isdir(save_dir):os.makedirs(save_dir)file_path = os.path.join(save_dir, "ner_results.txt")with open(file_path, "w", encoding="utf8") as fout:fout.write("\n".join(preds))# Print some examples# print(# "The results have been saved in the file: %s, some examples are shown below: "# % file_path)# print("\n".join(preds[:10]))with open("/home/aistudio/paddle-video-semantic-tag/data/nids.txt", "r") as inf:lines = inf.readlines()nid2ents = {}for entities, nid in zip(entities_list, lines):nid2ents[nid.strip()] = entitiessave_json = os.path.join(save_dir, "ents_results.json")with codecs.open(save_json, "w", encoding="utf-8") as ouf:json.dump(nid2ents, ouf, ensure_ascii=False)print("Saved " + save_json)args = Args() args.model_name_or_path = 'bert-wwm-ext-chinese' args.init_checkpoint_path = '/home/aistudio/paddle-video-semantic-tag/data/checkpoints/semantic_tag/model_567.pdparams' args.max_seq_length = 128 args.batch_size = 32 args.device = 'gpu'do_predict(args) [2022-04-29 02:13:12,761] [ INFO] - Found /home/aistudio/.paddlenlp/models/bert-wwm-ext-chinese/bert-wwm-ext-chinese-vocab.txt [2022-04-29 02:13:12,777] [ INFO] - Already cached /home/aistudio/.paddlenlp/models/bert-wwm-ext-chinese/bert-wwm-ext-chinese.pdparams 311it [00:13, 23.29it/s] Saved predict_results/ents_results.json

3. 結(jié)果文件生成和提交

運(yùn)行下列代碼后，將result.txt文件提交至比賽提交結(jié)果頁面即可。

In [6]

import os import os.path as osp import codecs import jsonclass Args():test_path = '/home/aistudio/dataset_sample/test_a.json'category_level1_result = '/home/aistudio/predict_results/level1_top1.json'category_level2_result = '/home/aistudio/predict_results/level2_top1.json'tag_result = '/home/aistudio/predict_results/ents_results.json'if __name__ == "__main__":args = Args()with codecs.open(args.test_path, "r", encoding="utf-8") as inf:print("Loading {}...".format(args.test_path))lines = inf.readlines()nids = [json.loads(line)["@id"] for line in lines]# load the prediction results of 'paddle-video-classify-tag' model on test-setwith codecs.open(args.category_level1_result, "r", encoding="utf-8") as inf:pred_level1 = json.load(inf)with codecs.open(args.category_level2_result, "r", encoding="utf-8") as inf:pred_level2 = json.load(inf)# load the prediction results of 'paddle-video-semantic-tag' model on test-setwith codecs.open(args.tag_result, "r", encoding="utf-8") as inf:pred_tags = json.load(inf)# merge results and generate an entry for each nid.submission_lines = []for nid in nids:level1_category = pred_level1[nid]["class_name"] \if nid in pred_level1 else ""level2_category = pred_level2[nid]["class_name"] \if nid in pred_level2 else ""tags = pred_tags[nid] if nid in pred_tags else []result = {"@id": nid,"category": [{"@meta": {"type": "level1"},"@value": level1_category},{"@meta": {"type": "level2"},"@value": level2_category},],"tag": [{"@value": tag} for tag in tags],}submission_lines.append(json.dumps(result, ensure_ascii=False) + "\n")with codecs.open("result.txt", "w", encoding="utf-8") as ouf:ouf.writelines(submission_lines)print("Saved result.txt") Loading /home/aistudio/dataset_sample/test_a.json... Saved result.txt

退出前務(wù)必執(zhí)行

運(yùn)行下列指令刪除臨時(shí)大文件，下次啟動(dòng)時(shí)重新生成，否則可能導(dǎo)致下次在線項(xiàng)目加載較慢。

In [7]

!rm -rf dataset_sample paddle-video-classify-tag paddle-video-semantic-tag predict_results tsn_features_test_a tsn_features_train_sample

請(qǐng)點(diǎn)擊此處查看本環(huán)境基本用法.
Please click?here?for more detailed instructions.

總結(jié)

以上是生活随笔為你收集整理的LIC 2022 视频语义理解基线（快速启动版）的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。