2 years ago

#63853

test-img

jing

clickhouse zookeeper node log_pointer node doesn't exist

clickhouse :) TRUNCATE TABLE <db_name>.<table_name> ON CLUSTER xxxx
┌─host────────┬─port─┬─status─┬─error─────────────────────────────────────────────────────────┬─num_hosts_remaining─┬─num_hosts_active─┐
│ xxx │ xxx │    341 │ Cannot execute replicated DDL query, maximum retries exceeded │                   5 │                5 │
└─────────────┴──────┴────────┴───────────────────────────────────────────────────────────────┴─────────────────────┴──────────────────┘
┌─host────────┬─port─┬─status─┬─error─────────────────────────────────────────────────────────┬─num_hosts_remaining─┬─num_hosts_active─┐
│ xxx │ xxx │    341 │ Cannot execute replicated DDL query, maximum retries exceeded │                   4 │                4 │
└─────────────┴──────┴────────┴───────────────────────────────────────────────────────────────┴─────────────────────┴──────────────────┘
┌─host────────┬─port─┬─status─┬─error─────────────────────────────────────────────────────────┬─num_hosts_remaining─┬─num_hosts_active─┐
│ xxx │ xx │    341 │ Cannot execute replicated DDL query, maximum retries exceeded │                   3 │                3 │
└─────────────┴──────┴────────┴───────────────────────────────────────────────────────────────┴─────────────────────┴──────────────────┘
┌─host────────┬─port─┬─status─┬─error─────────────────────────────────────────────────────────┬─num_hosts_remaining─┬─num_hosts_active─┐
│ xxx │ xxx │    341 │ Cannot execute replicated DDL query, maximum retries exceeded │                   2 │                1 │
│ xxx │ xxx │    341 │ Cannot execute replicated DDL query, maximum retries exceeded │                   1 │                1 │
└─────────────┴──────┴────────┴───────────────────────────────────────────────────────────────┴─────────────────────┴──────────────────┘
┌─host────────┬─port─┬─status─┬─error─────────────────────────────────────────────────────────┬─num_hosts_remaining─┬─num_hosts_active─┐
│ xxx │ xxx │    341 │ Cannot execute replicated DDL query, maximum retries exceeded │                   0 │                0 │
└─────────────┴──────┴────────┴───────────────────────────────────────────────────────────────┴─────────────────────┴──────────────────┘

log show:

2022.01.18 15:08:39.500203 [ 11583 ] {} <Error> mars.logout_local (ed49f1f3-5576-46bb-ad49-f1f35576d6bb): void DB::StorageReplicatedMergeTree::queueUpdatingTask(): Code: 999. Coordination::Exception: Can't get data for node /clickhouse/tables/0/xxx/xxx/replicas/xxx/log_pointer: node doesn't exist (No node). (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x944bdda in /usr/lib/debug/.build-id/66/99b86599a2121e78e0d42dd67791abd9ae5265.debug
1. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error, int) @ 0x11cf3795 in /usr/lib/debug/.build-id/66/99b86599a2121e78e0d42dd67791abd9ae5265.debug
2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Error) @ 0x11cf3a22 in /usr/lib/debug/.build-id/66/99b86599a2121e78e0d42dd67791abd9ae5265.debug
3. zkutil::ZooKeeper::get(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::shared_ptr<Poco::Event> const&) @ 0x11cfda12 in /usr/lib/debug/.build-id/66/99b86599a2121e78e0d42dd67791abd9ae5265.debug
4. DB::ReplicatedMergeTreeQueue::pullLogsToQueue(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>, DB::ReplicatedMergeTreeQueue::PullLogsReason) @ 0x117dc451 in /usr/lib/debug/.build-id/66/99b86599a2121e78e0d42dd67791abd9ae5265.debug
5. DB::StorageReplicatedMergeTree::queueUpdatingTask() @ 0x113ab28a in /usr/lib/debug/.build-id/66/99b86599a2121e78e0d42dd67791abd9ae5265.debug
6. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0x10898888 in /usr/lib/debug/.build-id/66/99b86599a2121e78e0d42dd67791abd9ae5265.debug
7. DB::BackgroundSchedulePool::threadFunction() @ 0x1089a8b7 in /usr/lib/debug/.build-id/66/99b86599a2121e78e0d42dd67791abd9ae5265.debug
8. void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, unsigned long, char const*)::$_1>(DB::BackgroundSchedulePool::BackgroundSchedulePool(unsigned long, unsigned long, char const*)::$_1&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) @ 0x1089b632 in /usr/lib/debug/.build-id/66/99b86599a2121e78e0d42dd67791abd9ae5265.debug
9. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x948ce7f in /usr/lib/debug/.build-id/66/99b86599a2121e78e0d42dd67791abd9ae5265.debug
10. void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda0'()> >(void*) @ 0x9490763 in /usr/lib/debug/.build-id/66/99b86599a2121e78e0d42dd67791abd9ae5265.debug
11. start_thread @ 0x7fa3 in /lib/x86_64-linux-gnu/libpthread-2.28.so
12. __clone @ 0xf94cf in /lib/x86_64-linux-gnu/libc-2.28.so


i go to zookeeper to find the node /clickhouse/tables/0/xxx/xxx/replicas/xxx/log_pointer.it not exists truly.but why?

select count(*) from <db_name>.<table_name>
┌─count()─┐
│  347279 │
└─────────┘

1 rows in set. Elapsed: 0.017 sec. 

clickhouse

0 Answers

Your Answer

Accepted video resources