Commit Graph

122 Commits

Author SHA1 Message Date
Abhishek Gola
199fd78571 Merge pull request #30351 from spicysharkco/fix/filestorage-release-state
core: fix FileStorage release leaks, state and unbalanced closing
2026-11-03 13:45:51 +05:30
spicysharkco
e147e535bd core: do not throw from FileStorage and WriteStructContext destructors
FileStorage::Impl::~Impl() calls release(), and release() can throw: for
example workaround() reports an empty key of a delayed Base64 sequence.
WriteStructContext::~WriteStructContext() calls endWriteStruct(), which
throws when the structure was already closed by hand or never opened. A
destructor is noexcept, so either exception ended the process.

Catch it in both destructors and log a warning instead, as other
destructors in OpenCV do. When release() throws in ~Impl(), call it a
second time, as FileStorage::open() does when opening fails: the first
call has already dropped the delayed structure that made it throw, so
the second one writes the rest of the file. ~Impl() then closes the
file in case both calls threw.
2026-10-21 22:14:06 +00:00
kevin9327
1b09dd45c9 core: read numbers with an upper-case exponent in FileStorage 2026-10-21 21:22:09 +09:00
spicysharkco
2d79dcf58e core: fix FileStorage release leaks, state and unbalanced closing
Impl::release() skipped the delayed struct (a sequence still waiting for
its first write in Base64 mode) when only the root was on the stack, so
its key leaked and the sequence was missing from the output. Resolve it
first. Reset the public FileStorage::state in release(),
releaseAndGetString() and a failed open(). An unbalanced endWriteStruct()
or closing bracket now throws and keeps the root on the stack; popping
the root made the next flush() read past the empty stack.
startNextStream() now resets the root in place instead of popping it.
A keyless writeRaw() of data into a map now throws in Base64 mode, as
it does without Base64. At the top level it opened a Base64 writer that
was never freed, and in a nested map after a closed structure it wrote
output that could not be read back.

Fixes #30338
2026-10-16 22:06:31 +00:00
Alexander Smorkalov
af1e2232cd Merge pull request #29368 from tonuonu:fix-filestorage-read-bigint-29363
core: FileStorage reads integers above INT_MAX into float/double without truncation 🧑‍💻🤖
2026-06-23 15:38:59 +03:00
Tonu Samuel
dceead809e core: FileStorage reads integers above INT_MAX into float/double without truncation
FileNode::operator double() and operator float() read an INT node via readInt()
(32-bit), truncating values above INT_MAX -- e.g. an integer 6662329666 from an
externally-produced json/yaml/xml is read back as -1927604926. The node stores
the value as int64 (operator int64_t() already reads it correctly via readLong),
so use readLong() for the floating-point conversions too. Values that fit in
int32 are unchanged (sign-extended); larger ones are now correct.

Reader side of #29363 (the writer side was #29364).
2026-06-23 11:15:11 +03:00
uwezkhan
fc746f35b9 fix fmt_pairs stack overflow in calcElemSize and decodeSimpleFormat 2026-06-20 15:31:34 +05:30
uwezkhan
2906d4d73a reject nd-matrix dim count above CV_MAX_DIM in FileStorage read 2026-06-14 02:51:20 +05:30
Kumataro
8449b9e468 core: safe handling of compressed file level extension (.gz[0-9])
- Replace unsafe pointer arithmetic and direct buffer modification with std::string methods.
- Update documentation to clarify that the last digit is used as compression level and truncated from the actual filename.
- Add test cases for .gz and .gz0-9
2026-05-05 07:41:59 +09:00
Matt Van Horn
8e1fa1bbdc Merge pull request #28620 from mvanhorn:osc/28619-fix-yaml-parsekey-empty-key-oob
core: fix heap-buffer-overflow in YAML parseKey for empty keys #28620

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake

### Description

Fixes https://github.com/opencv/opencv/issues/28619

Moves the "empty key" check before the backward do-while scan in `YAMLParser::parseKey()`.

**Problem:** When parsing a YAML mapping with an empty key (e.g. `: 10` at column 0), `endptr == ptr` after the forward scan finds `:`. The do-while loop `do c = *--endptr; while(c == ' ')` always executes at least once, so it decrements `endptr` to `ptr-1` and reads one byte before the heap allocation (ASan: heap-buffer-overflow READ of size 1).

**Fix:** Check `endptr == ptr` before entering the backward loop. If the key is empty, raise `CV_PARSE_ERROR_CPP("An empty key")` immediately without the OOB read.

This contribution was developed with AI assistance (Claude Code).
2026-03-12 10:37:24 +03:00
Yuantao Feng
07cf36cbb0 Merge pull request #27587 from fengyuentau:4x/core/filestorage_json_support_backslash
core: support parsing back slash \ in parseKey in FileStorage (JSON) #27587

Fixes #27585 

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2025-07-29 12:04:55 +03:00
Yuantao Feng
603d7ded88 Merge pull request #27579 from fengyuentau:4x/core/filestorage
core: support parsing null in json parser in FileStorage #27579

Fixes https://github.com/opencv/opencv/issues/27578

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
2025-07-28 08:59:46 +03:00
MaximSmolskiy
4d23b56d98 Refactor test for FileStorage Base64 2025-02-09 01:38:14 +03:00
Maxim Smolskiy
08a24ba2cf Merge pull request #26846 from MaximSmolskiy:fix_bug_with_int64_support_for_FileStorage
Fix bug with int64 support for FileStorage #26846

### Pull Request Readiness Checklist

Fix #26829, https://github.com/opencv/opencv-python/issues/1078

In current implementation of `int64` support raw size of recorded integer is variable (`4` or `8` bytes depending on value). But then we iterate over nodes we need to know it exact value
dfad11aae7/modules/core/src/persistence.cpp (L2596-L2609)

Bug is that `rawSize` method still return `4` for any integer. I haven't figured out a way how to get variable raw size for integer in this method. I made raw size for integer is constant and equal to `8`.

Yes, after this patch memory consumption for integers will increase, but I don't know a better way to do it yet. At least this fixes bug and implementation becomes more correct

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2025-01-28 11:08:26 +03:00
Dmitry Kurtaev
c230841105 Merge pull request #26446 from dkurt:file_storage_empty_and_1d_mat
* Change style of empty and 1d Mat in FileStorage

* Remove misleading 1d Mat test
2024-11-12 17:51:10 +03:00
Dmitry Kurtaev
37c2af63f0 Merge pull request #26434 from dkurt:dk/int64_file_storage_4.x
int64 data type support for FileStorage. 1d and empty Mat with exact dimensions #26434

### Pull Request Readiness Checklist

Port of https://github.com/opencv/opencv/pull/26399 to 4.x branch

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2024-11-11 14:13:33 +03:00
Kumataro
40428d919d Merge pull request #26259 from Kumataro:fix26258
core: C-API cleanup: RNG algorithms in core(4.x) #26259

- replace CV_RAND_UNI and NORMAL to cv::RNG::UNIFORM and cv::RNG::NORMAL.

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2024-10-08 15:55:00 +03:00
Kumataro
be3c519956 core: FileStorage: detect invalid attribute value 2024-07-26 05:55:00 +09:00
Kumataro
d22d0bd49c core: persistence: use hfloat instead of float16_t 2024-04-11 05:18:25 +09:00
Kumataro
b14ea19466 Merge pull request #25351 from Kumataro:fix25073_format_g
core: persistence: output reals as human-friendly expression. #25351

Close #25073
Related https://github.com/opencv/opencv/pull/25087

This patch is need to merge same time with https://github.com/opencv/opencv_contrib/pull/3714

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
2024-04-10 15:17:15 +03:00
Maksim Shabunin
1bccc14e05 Merge pull request #24343 from mshabunin:fix-test-writes
Fix tests writing to current work dir #24343

Several tests were writing files in the current work directory and did not clean up after test. Moved all temporary files to the `/tmp` dir and added a cleanup code.
2023-10-03 16:34:25 +03:00
Sean McBride
35f1a90df7 Merge pull request #22149 from seanm:sprintf
Replaced sprintf with safer snprintf

* Straightforward replacement of sprintf with safer snprintf

* Trickier replacement of sprintf with safer snprintf

Some functions were changed to take another parameter: the size of the buffer, so that they can pass that size on to snprintf.
2022-06-25 06:48:22 +03:00
rogday
7daf84fb44 address security concerns in persistence 2022-04-14 22:18:00 +03:00
Alexander Alekhin
5ba9a089e1 core(persistence): avoid NULL pointer dereference 2022-01-18 04:56:43 +00:00
Smirnov Egor
b42623ff9d port base64 encoding from 3.4 2021-07-08 10:42:44 +03:00
Alexander Alekhin
4eac198270 core(persistence): fix types format handling, fix 16F support 2021-06-29 11:26:57 +00:00
Felix Gorschlüter
c996fd1c06 Small improvements to persistence-API doc 2020-11-20 10:49:51 +03:00
Alexander Alekhin
ffe0d50447 core(persistence): fix "use after free" bug
- do not store user-controlled "FileStorage" pointer
- store FileStorage::Impl pointer instead
2020-07-17 21:39:06 +00:00
Vadim Pisarevsky
4564b8a224 Merge pull request #17841 from vpisarev:fixed_fs_dtor
* fixed issue #17412

* Update test_io.cpp
2020-07-17 02:28:50 +00:00
Alexander Alekhin
673eb2b006 core(persistence): fix resource leaks - force closing files 2020-03-24 11:38:09 +00:00
Vadim Pisarevsky
3efa78311a Merge pull request #16488 from vpisarev:filestorage_longlines
trying to fix handling file storages with extremely long lines

* trying to fix handling of file storages with extremely long lines: https://github.com/opencv/opencv/issues/11061

* * fixed errorneous pointer access in JSON parser.
* it's now crash-test time! temporarily set the initial parser buffer size to just 40 bytes. let's run all the test and check if the buffer is always correctly resized and handled

* fixed pointer use in JSON parser; added the proper test to catch this case

* fixed the test to make it more challenging. generate test json with
*
**
***
etc. shape
2020-02-11 18:46:15 +03:00
Alexander Smorkalov
bd6ab6d38e Implemented BASE64 support in cv::FileStorage parser. 2020-02-10 11:38:19 +03:00
Alexander Alekhin
dcf72e49e2 core(persistence): fix processing of multiple documents 2019-11-05 18:28:15 +03:00
TH3CHARLie
a165f55579 Merge pull request #15842 from TH3CHARLie:yaml-fix
* fix yaml parse

* add test for YAML parse multiple documents

* remove trailing whitespace in test
2019-11-04 16:27:48 +03:00
Alexander Alekhin
e2a5a6a05c Merge remote-tracking branch 'upstream/3.4' into merge-3.4 2019-09-25 18:32:44 +00:00
Alexander Alekhin
fcc69d5a60 core(test): fix check conditions 2019-09-22 11:28:41 +00:00
Alexander Alekhin
d4501f08b8 core(persistence): disable base64 tests due missing encoders 2019-07-16 15:37:58 +03:00
Alexander Alekhin
2df7736562 Merge branch 'core_fix_base64_packed_struct' 2019-07-16 15:35:43 +03:00
Alexander Alekhin
4ea8526e9f core(persistence): fix writeRaw() / readRaw() struct support
- writeRaw(): support structs
- readRaw(): 'len' is buffer limit in bytes (documentation is fixed)
2019-07-16 14:03:39 +03:00
Alexander Alekhin
c3b838b738 core(persistence): struct storage layout without alignment gaps 2019-07-15 21:37:20 +00:00
Dmitry Kurtaev
6c76c8f881 Add a test for FileNode::keys() 2018-11-12 13:38:18 +03:00
Dmitry Kurtaev
a7f30391d0 Add FileNode::keys() missed during refactoring 2018-11-12 13:34:01 +03:00
Matt Bennett
10e1c2b3cd Merge pull request #11935 from mattmyne:filestorage_matx
* Support for Matx read/write by FileStorage

* Only empty filestorage read now produces default Matx. Split Matx IO test into smaller units. Test checks for exception thrown if reading a Mat into a Matx of different size.
2018-11-09 20:05:59 +03:00
Vadim Pisarevsky
0f622206e4 completely new C++ persistence implementation (#13011)
* integrated the new C++ persistence; removed old persistence; most of OpenCV compiles fine! the tests have not been run yet

* fixed multiple bugs in the new C++ persistence

* fixed raw size of the parsed empty sequences

* [temporarily] excluded obsolete applications traincascade and createsamples from build

* fixed several compiler warnings and multiple test failures

* undo changes in cocoa window rendering (that was fixed in another PR)

* fixed more compile warnings and the remaining test failures (hopefully)

* trying to fix the last little warning
2018-11-02 00:27:06 +03:00
Maksim Shabunin
e0f524d3b7 Fixed several incorrect printf format specifiers 2018-09-24 11:31:40 +03:00
Pavel Rojtberg
d1c842cf29 core: fix printf warnings by using c++11 format 2018-09-24 11:31:40 +03:00
Alexander Alekhin
df8b057b44 avoid Ptr<> == NULL checks 2018-09-09 19:30:46 +00:00
Alexander Alekhin
8a3c394d6a don't use constructors for C API structures 2018-09-06 14:34:16 +03:00
Alexander Alekhin
703f79b757 tests: add "bigdata" tests 2018-05-15 15:56:26 +03:00
LaurentBerger
03eb463f1c Solves #11156 (#11160)
* Solves #11156

* Check file size for all file format. Disable APPEND if file is empty

* Add test for APPEND mode
2018-03-28 15:35:38 +03:00