Skip to main content

Semantic Indexing of Large Scale Video Archives

Project Member(s): Yang, Y.

Funding or Partner Organisation: D2D CRC LTD

Start year: 2015

Summary: Identifying semantic content in Internet videos has long been a goal of multimedia analysis and retrieval, and has broad impact on many real world applications, ranging from our daily life to security. This fundamental problem is also a building block for many tasks such as visualization, abstraction, recommendation, retrieval and summarization of videos. As opposed to merely using user-generated metadata, such as titles and descriptions, content-based semantic indexing strives to leverage concepts that are automatically detected in the video, such as objects/scenes/actions. This semantic indexing relies on extensive video understanding, and requires neither metadata nor example videos. The outcome of this project will provide us hassle free analytical tools for big video data management and utilization. The output of our system can also be feed into other text data mining platforms to enrich the type of data source for powerful multi-source multi-modality data mining at large scale.

Publications:

Nie, L, Wei, X, Zhang, D, Wang, X, Gao, Z & Yang, Y 2017, 'Data-driven answer selection in community QA systems', IEEE Transactions on Knowledge and Data Engineering, vol. 29, no. 6, pp. 1186-1198.
View/Download from: UTS OPUS or Publisher's site

Yan, Y, Yang, T, Yang, Y & Chen, J 2017, 'A Framework of Online Learning with Imbalanced Streaming Data', Proceedings of the Thirty-Firs AAAI Conference on Artificial Intelligence, AAAI Conference on Artificial Intelligence, AAAI, San Francisco, USA, pp. 2817-2823.
View/Download from: UTS OPUS

Chang, X, Nie, F, Yang, Y, Zhang, C & Huang, H 2016, 'Convex Sparse PCA for Unsupervised Feature Learning', ACM Transactions on Knowledge Discovery from Data, vol. 11, no. 1, pp. 1-16.
View/Download from: UTS OPUS or Publisher's site

Gan, C, Yang, Y, Zhu, L, Zhao, D & Zhuang, Y 2016, 'Recognizing an Action Using Its Name: A Knowledge-Based Approach', International Journal of Computer Vision, vol. 120, no. 1, pp. 61-77.
View/Download from: UTS OPUS or Publisher's site

Chang, X, Yu, YL, Yang, Y & Xing, EP 2016, 'They are not equally reliable: Semantic event search using differentiated concept classifiers', Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, Nevada, United States, pp. 1884-1893.
View/Download from: UTS OPUS or Publisher's site

Pan, P, Xu, Z, Yang, Y, Wu, F & Zhuang, Y 2016, 'Hierarchical recurrent neural encoder for video representation with application to captioning', Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Seattle, WA, USA, pp. 1029-1038.
View/Download from: UTS OPUS or Publisher's site

Chang, X, Nie, F, Yang, Y & Huang, H, 'Improved Spectral Clustering via Embedded Label Propagation'.
View/Download from: UTS OPUS

Keywords: Video Analysis, Semantic Indexing, Computer Vision, Multimedia

FOR Codes: Artificial Intelligence and Image Processing, Pattern Recognition and Data Mining, Multimedia Programming, Information Processing Services (incl. Data Entry and Capture), Application Tools and System Utilities, Electronic Information Storage and Retrieval Services