UTF-8 CPP的使用

UTF-8 CPP是一个简单、小巧、轻量级、跨平台的UTF-8编码字符串库。

下面对其使用方法进行简单的介绍：

1. 从http://sourceforge.net/projects/utfcpp/下载最新的utf8_v2_3_4.zip源码，将其解压缩；

2. 新建一个vs2013 控制台工程TestUTF-8CPP，将utf-8cpp中的src文件加入到包含目录中；

3. 参考http://utfcpp.sourceforge.net/，测试代码内容为：

#include "stdafx.h"
#include <iostream>
#include <string>
#include <fstream>
#include <vector>
#include <assert.h>
#include "utf8.h"// checks whether the content of a file is valid UTF-8 encoded text without reading the content into the memory
bool valid_utf8_file(const char* file_name)
{std::ifstream ifs(file_name);if (!ifs)return false; // even better, throw herestd::istreambuf_iterator<char> it(ifs.rdbuf());std::istreambuf_iterator<char> eos;return utf8::is_valid(it, eos);
}// The function will replace any invalid UTF-8 sequence with a Unicode replacement character
void fix_utf8_string(std::string& str)
{std::string temp;utf8::replace_invalid(str.begin(), str.end(), back_inserter(temp));str = temp;
}int main(int argc, char* argv[])
{const char* test_file_path = "../../../demo/test.txt";// Open the test file(contains UTF-8 encoded text)std::ifstream fs8(test_file_path);if (!fs8.is_open()) {std::cout << "Could not open " << test_file_path << std::endl;return -1;}if (!valid_utf8_file(test_file_path))return -1;unsigned line_count = 1;std::string line;// Play with all the lines in the filewhile (getline(fs8, line)) {// check for invalid utf-8 (for a simple yes/no check, there is also utf8::is_valid function)std::string::iterator end_it = utf8::find_invalid(line.begin(), line.end());if (end_it != line.end()) {std::cout << "Invalid UTF-8 encoding detected at line " << line_count << "\n";std::cout << "This part is fine: " << std::string(line.begin(), end_it) << "\n";}// Get the line length (at least for the valid part)int length = utf8::distance(line.begin(), end_it);std::cout << "Length of line " << line_count << " is " << length << "\n";// Convert it to utf-16std::vector<unsigned short> utf16line;utf8::utf8to16(line.begin(), end_it, back_inserter(utf16line));// And back to utf-8std::string utf8line;utf8::utf16to8(utf16line.begin(), utf16line.end(), back_inserter(utf8line));// Confirm that the conversion went OK:if (utf8line != std::string(line.begin(), end_it))std::cout << "Error in UTF-16 conversion at line: " << line_count << "\n";line_count++;}std::string str = "ABCD";std::vector<unsigned short> utf16result;utf8::utf8to16(str.begin(), str.end(), std::back_inserter(utf16result));size_t size1 = utf16result.size();std::string str2 = "濦粿夿旴";std::string utf8str;utf8::utf16to8(str2.begin(), str2.end(), std::back_inserter(utf8str));size_t size2 = utf8str.length();char utf8_with_surrogates[] = "\xe6\x97\xa5\xd1\x88\xf0\x9d\x84\x9e";std::vector <unsigned short> utf16result1;utf8::utf8to16(utf8_with_surrogates, utf8_with_surrogates + 9, back_inserter(utf16result1));assert(utf16result1.size() == 4);assert(utf16result1[2] == 0xd834);assert(utf16result1[3] == 0xdd1e);unsigned short utf16string[] = { 0x41, 0x0448, 0x65e5, 0xd834, 0xdd1e };std::vector<unsigned char> utf8result;utf8::utf16to8(utf16string, utf16string + 5, back_inserter(utf8result));assert(utf8result.size() == 10);char* szSex = "\xe7\x94\xb7\x00";std::basic_string<wchar_t> sex;utf8::utf8to16(szSex, szSex + strlen(szSex), back_inserter(sex));if (sex != L"男") {std::cout << "unicode char utf16 error" << std::endl;return -1;}std::cout << "ok!" << std::endl;return 0;
}

GitHub：https://github.com/fengbingchun/UTF-8CPP_Test

UTF-8 CPP的使用相关推荐

Zip.cpp的例子
. 简介我已经介绍了在{A} XZip.本文介绍XZip也XUnzip,这允许您添加ZIP解压缩到您的应用程序,而不使用任何的.lib或.dll. 首先,让我承认的工作和生产的.cpp和.h文件是基 ...
C++ .h(头文件) 与 .cpp(源文件) 的使用
.h 文件: .h是头文件 ,里面主要是写类的声明(包括类里面的成员和方法的声明).函数原型.#define常数等, 注意.h文件写的时候有特定的格式就是开头和结尾 #ifndef TEST_HEAD ...
利用VS+MFC+Opencv显示图像和视频所需添加类（CvvImage.h和CvvImage.cpp的源码）。
CvvImage.h代码: #pragma once #ifndef CVVIMAGE_CLASS_DEF #define CVVIMAGE_CLASS_DEF #include "open ...
拇指接龙游戏升级记录03(升级MainScene.cpp)
MainScene是拇指接龙游戏的主游戏场景文件,拥有近5000行代码. 说实在的,实现自cocos2d-x 2.x版本向当下最新的3.8.1版本的升级过程,其中涉及的技术不是一下能够说明的.有些是形 ...
GATB的使用小例子test.cpp
1.touch test.cpp,,文件夹中出现test.cpp touch test.cpp 2. test.cpp的内容 #include <gatb/gatb_core.hpp>i ...
Linux下运行.cpp文件
如何在Ubuntu16下运行一个简单的.cpp文件呢,做法如下: 假设我在桌面上写了一个hell,world程序; 保存为abc.cpp 然后在终端打开: 1.点击保存的文件的属性,看在哪里,我的是在 ...
2020-10-26runtime error: member access within null pointer of type ‘struct ListNode‘ (solution.cpp)错
runtime error: member access within null pointer of type 'struct ListNode' (solution.cpp)错误 /*** Def ...
matlab怎么跑.cpp程序,MATLAB编译cpp文件
目的:打通MATLAB与VC之间的通道,实现用MATLAB调用VC程序,以及VC调用MATLAB程序. 上篇博客实现了将MATLAB的M文件编译成C/C++文件,exe应用程序.这篇实现MATLAB编 ...
python如何调用cpp文件的接口函数_C++中嵌入Python调用
python嵌入到C++中把python嵌入的C++里面需要做一些步骤安装python程序,这样才能使用python的头文件和库在我们写的源文件中增加"Python.h"头文 ...

UTF-8 CPP的使用

UTF-8 CPP的使用相关推荐

最新文章

热门文章